[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:02:19 dist-test-slave-7153 krb5kdc[9716](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:02:19 dist-test-slave-7153 krb5kdc[9716](info): set up 2 sockets
Sep 26 15:02:19 dist-test-slave-7153 krb5kdc[9716](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260926 15:02:20.153193 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.967s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:02:20 dist-test-slave-7153 krb5kdc[9716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:02:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:02:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:02:20.318399 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:41577
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:35099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:41577
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:20.646114 9729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:20.646423 9729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:20.646533 9729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:20.653924 9729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:02:20.654042 9729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:20.654103 9729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:20.654181 9729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:02:20.654245 9729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:02:20.664120 9729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:35099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:41577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:41577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:20.665846 9729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:20.667271 9729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:20.679702 9737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:20.679774 9729 server_base.cc:1034] running on GCE node
W20260926 15:02:20.679752 9735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:20.679805 9734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:20.680807 9729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:20.682315 9729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:20.683538 9729 hybrid_clock.cc:648] HybridClock initialized: now 1790434940683482 us; error 43 us; skew 500 ppm
I20260926 15:02:20.683881 9729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:20 dist-test-slave-7153 krb5kdc[9716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434940, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:20.686882 9729 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:02:20.688413 9729 webserver.cc:460] Webserver started at http://127.9.123.126:41549/ using document root <none> and password file <none>
I20260926 15:02:20.689059 9729 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:20.689200 9729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:20.689491 9729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:20.692416 9729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "86f71e31d35244008c6ecf66cbf878c8"
format_stamp: "Formatted at 2026-09-26 15:02:20 on dist-test-slave-7153"
server_key: "7a9cbecd3fa73462a09b41579c83b029"
server_key_iv: "99da37df3c1d2f2abc4b04e7193837d1"
server_key_version: "encryptionkey@0"
I20260926 15:02:20.693181 9729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "86f71e31d35244008c6ecf66cbf878c8"
format_stamp: "Formatted at 2026-09-26 15:02:20 on dist-test-slave-7153"
server_key: "7a9cbecd3fa73462a09b41579c83b029"
server_key_iv: "99da37df3c1d2f2abc4b04e7193837d1"
server_key_version: "encryptionkey@0"
I20260926 15:02:20.698426 9729 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260926 15:02:20.701826 9744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:20.703188 9729 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:02:20.703367 9729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "86f71e31d35244008c6ecf66cbf878c8"
format_stamp: "Formatted at 2026-09-26 15:02:20 on dist-test-slave-7153"
server_key: "7a9cbecd3fa73462a09b41579c83b029"
server_key_iv: "99da37df3c1d2f2abc4b04e7193837d1"
server_key_version: "encryptionkey@0"
I20260926 15:02:20.703557 9729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:20.720511 9729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:20.724777 9729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:20.725080 9729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:20.744570 9729 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:41577
I20260926 15:02:20.744572 9795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:41577 every 8 connection(s)
I20260926 15:02:20.745954 9729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:02:20.749794 9796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:20.755484 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 9729
I20260926 15:02:20.755752 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:02:20.757587 9709 external_mini_cluster.cc:1428] Setting key 50b694e7158d1e488ab16b7db6a99a03
I20260926 15:02:20.759794 9796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86f71e31d35244008c6ecf66cbf878c8: Bootstrap starting.
I20260926 15:02:20.763103 9796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86f71e31d35244008c6ecf66cbf878c8: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:20.764113 9796 log.cc:824] T 00000000000000000000000000000000 P 86f71e31d35244008c6ecf66cbf878c8: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:20.766930 9796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86f71e31d35244008c6ecf66cbf878c8: No bootstrap required, opened a new log
I20260926 15:02:20.775460 9796 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86f71e31d35244008c6ecf66cbf878c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f71e31d35244008c6ecf66cbf878c8" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 41577 } }
I20260926 15:02:20.775942 9796 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86f71e31d35244008c6ecf66cbf878c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:20.776057 9796 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86f71e31d35244008c6ecf66cbf878c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86f71e31d35244008c6ecf66cbf878c8, State: Initialized, Role: FOLLOWER
I20260926 15:02:20.776674 9796 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86f71e31d35244008c6ecf66cbf878c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f71e31d35244008c6ecf66cbf878c8" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 41577 } }
I20260926 15:02:20.776876 9796 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86f71e31d35244008c6ecf66cbf878c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:02:20.776997 9796 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86f71e31d35244008c6ecf66cbf878c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:02:20.777189 9796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86f71e31d35244008c6ecf66cbf878c8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:20.779894 9796 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86f71e31d35244008c6ecf66cbf878c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f71e31d35244008c6ecf66cbf878c8" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 41577 } }
I20260926 15:02:20.780505 9796 leader_election.cc:302] T 00000000000000000000000000000000 P 86f71e31d35244008c6ecf66cbf878c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86f71e31d35244008c6ecf66cbf878c8; no voters:
I20260926 15:02:20.781191 9796 leader_election.cc:288] T 00000000000000000000000000000000 P 86f71e31d35244008c6ecf66cbf878c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:02:20.781314 9801 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86f71e31d35244008c6ecf66cbf878c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:20.782353 9801 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86f71e31d35244008c6ecf66cbf878c8 [term 1 LEADER]: Becoming Leader. State: Replica: 86f71e31d35244008c6ecf66cbf878c8, State: Running, Role: LEADER
I20260926 15:02:20.782996 9801 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86f71e31d35244008c6ecf66cbf878c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f71e31d35244008c6ecf66cbf878c8" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 41577 } }
I20260926 15:02:20.783321 9796 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86f71e31d35244008c6ecf66cbf878c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:02:20.785758 9802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86f71e31d35244008c6ecf66cbf878c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86f71e31d35244008c6ecf66cbf878c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f71e31d35244008c6ecf66cbf878c8" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 41577 } } }
I20260926 15:02:20.786617 9802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86f71e31d35244008c6ecf66cbf878c8 [sys.catalog]: This master's current role is: LEADER
Sep 26 15:02:20 dist-test-slave-7153 krb5kdc[9716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:20.792079 9803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86f71e31d35244008c6ecf66cbf878c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86f71e31d35244008c6ecf66cbf878c8. Latest consensus state: current_term: 1 leader_uuid: "86f71e31d35244008c6ecf66cbf878c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f71e31d35244008c6ecf66cbf878c8" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 41577 } } }
I20260926 15:02:20.792986 9803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86f71e31d35244008c6ecf66cbf878c8 [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:20.799165 9806 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:02:20.806291 9806 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:02:20.809654 9798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:20.761400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35784 (local address 127.9.123.126:41577)
0926 15:02:20.762274 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:20.762289 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:20.763738 (+ 1449us) server_negotiation.cc:407] Connection header received
0926 15:02:20.774805 (+ 11067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:20.774836 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:20.775358 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:20.775821 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:20.779220 (+ 3399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:20.781259 (+ 2039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:20.783783 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:20.784984 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:20.785218 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:20.794229 (+ 9011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:20.794243 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:20.794270 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:20.794330 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:20.798916 (+ 4586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:20.800395 (+ 1479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:20.800409 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:20.800422 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:20.800688 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:20.801361 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:20.801367 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:20.801371 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:20.802314 (+ 943us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:20.806592 (+ 4278us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:20.806956 (+ 364us) server_negotiation.cc:299] Negotiation successful
0926 15:02:20.807692 (+ 736us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":385,"threads_started":1}
I20260926 15:02:20.820168 9806 catalog_manager.cc:1347] Generated new cluster ID: fe403a249d124635b7457cc021beb0f0
I20260926 15:02:20.820297 9806 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:02:20.843467 9806 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:02:20.844897 9806 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:02:20.860003 9806 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86f71e31d35244008c6ecf66cbf878c8: Generated new TSK 0
I20260926 15:02:20.860877 9806 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.123.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
I20260926 15:02:20.902381 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:41577
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:35099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:21.252873 9822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:21.253227 9822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:21.253382 9822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:21.261214 9822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:21.261332 9822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:21.261489 9822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:02:21.271560 9822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:35099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:41577
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:21.272904 9822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:21.274238 9822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:21.287551 9827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:21.287573 9830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:21.287551 9828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:21.288524 9822 server_base.cc:1034] running on GCE node
I20260926 15:02:21.289130 9822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:21.290119 9822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:21.291335 9822 hybrid_clock.cc:648] HybridClock initialized: now 1790434941291278 us; error 46 us; skew 500 ppm
I20260926 15:02:21.291669 9822 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:21 dist-test-slave-7153 krb5kdc[9716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434941, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:21.294852 9822 init.cc:377] Logged in from keytab as kudu/127.9.123.65@KRBTEST.COM (short username kudu)
I20260926 15:02:21.296411 9822 webserver.cc:460] Webserver started at http://127.9.123.65:36573/ using document root <none> and password file <none>
I20260926 15:02:21.297564 9822 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:21.297695 9822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:21.298008 9822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:21.301136 9822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "62a9e4a3db35425c81305de5da1d999b"
format_stamp: "Formatted at 2026-09-26 15:02:21 on dist-test-slave-7153"
server_key: "0a41cfd354e1f18942f73d413c4ccc3a"
server_key_iv: "b9da3e459d11b4aa41a8e55af5e64887"
server_key_version: "encryptionkey@0"
I20260926 15:02:21.301905 9822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "62a9e4a3db35425c81305de5da1d999b"
format_stamp: "Formatted at 2026-09-26 15:02:21 on dist-test-slave-7153"
server_key: "0a41cfd354e1f18942f73d413c4ccc3a"
server_key_iv: "b9da3e459d11b4aa41a8e55af5e64887"
server_key_version: "encryptionkey@0"
I20260926 15:02:21.307327 9822 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260926 15:02:21.310981 9837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:21.312480 9822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 15:02:21.312661 9822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "62a9e4a3db35425c81305de5da1d999b"
format_stamp: "Formatted at 2026-09-26 15:02:21 on dist-test-slave-7153"
server_key: "0a41cfd354e1f18942f73d413c4ccc3a"
server_key_iv: "b9da3e459d11b4aa41a8e55af5e64887"
server_key_version: "encryptionkey@0"
I20260926 15:02:21.312867 9822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:21.333871 9822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:21.338203 9822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:21.338478 9822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:21.339876 9822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:02:21.341320 9822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:21.341400 9822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:21.341521 9822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:21.341570 9822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:21.376456 9822 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:42425
I20260926 15:02:21.376524 9949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:42425 every 8 connection(s)
I20260926 15:02:21.377807 9822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:02:21.380908 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 9822
I20260926 15:02:21.381148 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:02:21.381721 9709 external_mini_cluster.cc:1428] Setting key 206be5f97ecbdba368dd176b1666e610
Sep 26 15:02:21 dist-test-slave-7153 krb5kdc[9716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434941, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.123.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.66@KRBTEST.COM" created.
I20260926 15:02:21.400799 9952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:21.380365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:54063 (local address 127.9.123.126:41577)
0926 15:02:21.380887 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:21.380893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:21.381955 (+ 1062us) server_negotiation.cc:407] Connection header received
0926 15:02:21.384098 (+ 2143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:21.384103 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:21.384186 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:21.384307 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:21.386174 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:21.387682 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:21.389583 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:21.390088 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:21.390206 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:21.394561 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:21.394566 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:21.394569 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:21.394605 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:21.397258 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:21.398226 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:21.398231 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:21.398234 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:21.398362 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:21.398859 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:21.398863 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:21.398865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:21.399163 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:21.399283 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:21.400214 (+ 931us) server_negotiation.cc:299] Negotiation successful
0926 15:02:21.400466 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":236,"threads_started":1}
I20260926 15:02:21.401989 9953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:21.380942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:41577 (local address 127.9.123.65:54063)
0926 15:02:21.381767 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:21.381820 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:21.383826 (+ 2006us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:21.384674 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:21.384685 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:21.385313 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:21.385967 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:21.385988 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:21.387876 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:21.387880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:21.389382 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:21.389400 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:21.390352 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:21.390359 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:21.390702 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:21.391477 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:21.391505 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:21.394261 (+ 2756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:21.397423 (+ 3162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:21.397433 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:21.397456 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:21.398095 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:21.398450 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:21.398453 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:21.398456 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:21.398729 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:21.399297 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:21.399311 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:21.399785 (+ 474us) client_negotiation.cc:769] Sending connection context
0926 15:02:21.400061 (+ 276us) client_negotiation.cc:240] Negotiation successful
0926 15:02:21.400739 (+ 678us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":326,"threads_started":1}
I20260926 15:02:21.403559 9950 heartbeater.cc:347] Connected to a master server at 127.9.123.126:41577
I20260926 15:02:21.403924 9950 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:21.404769 9950 heartbeater.cc:511] Master 127.9.123.126:41577 requested a full tablet report, sending...
I20260926 15:02:21.407044 9761 ts_manager.cc:194] Registered new tserver with Master: 62a9e4a3db35425c81305de5da1d999b (127.9.123.65:42425)
I20260926 15:02:21.408543 9761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.65@KRBTEST.COM'} at 127.9.123.65:54063
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
I20260926 15:02:21.420432 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:41577
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:35099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:21.763177 9956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:21.763481 9956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:21.763607 9956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:21.771286 9956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:21.771431 9956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:21.771585 9956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:02:21.782155 9956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:35099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:41577
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:21.783553 9956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:21.785044 9956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:21.799321 9961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:21.799239 9962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:21.799273 9964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:21.800693 9956 server_base.cc:1034] running on GCE node
I20260926 15:02:21.801462 9956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:21.802520 9956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:21.803754 9956 hybrid_clock.cc:648] HybridClock initialized: now 1790434941803706 us; error 51 us; skew 500 ppm
I20260926 15:02:21.804098 9956 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:21 dist-test-slave-7153 krb5kdc[9716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434941, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:21.808200 9956 init.cc:377] Logged in from keytab as kudu/127.9.123.66@KRBTEST.COM (short username kudu)
I20260926 15:02:21.809898 9956 webserver.cc:460] Webserver started at http://127.9.123.66:40695/ using document root <none> and password file <none>
I20260926 15:02:21.810667 9956 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:21.810781 9956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:21.811057 9956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:21.814607 9956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "4262af25a9f3425392b3914625a92945"
format_stamp: "Formatted at 2026-09-26 15:02:21 on dist-test-slave-7153"
server_key: "51e1ab9d82364258c37eb410b991ea46"
server_key_iv: "9c432e85ea20bbc5352458a2c7b00c62"
server_key_version: "encryptionkey@0"
I20260926 15:02:21.815511 9956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "4262af25a9f3425392b3914625a92945"
format_stamp: "Formatted at 2026-09-26 15:02:21 on dist-test-slave-7153"
server_key: "51e1ab9d82364258c37eb410b991ea46"
server_key_iv: "9c432e85ea20bbc5352458a2c7b00c62"
server_key_version: "encryptionkey@0"
I20260926 15:02:21.821784 9956 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260926 15:02:21.825990 9971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:21.827646 9956 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 15:02:21.827862 9956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "4262af25a9f3425392b3914625a92945"
format_stamp: "Formatted at 2026-09-26 15:02:21 on dist-test-slave-7153"
server_key: "51e1ab9d82364258c37eb410b991ea46"
server_key_iv: "9c432e85ea20bbc5352458a2c7b00c62"
server_key_version: "encryptionkey@0"
I20260926 15:02:21.828068 9956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:21.872474 9956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:21.876875 9956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:21.877197 9956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:21.878189 9956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:02:21.879588 9956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:21.879663 9956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:21.879786 9956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:21.879835 9956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:21.913623 9956 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:45071
I20260926 15:02:21.913683 10083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:45071 every 8 connection(s)
I20260926 15:02:21.914927 9956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:02:21.917718 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 9956
I20260926 15:02:21.917896 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:02:21.918399 9709 external_mini_cluster.cc:1428] Setting key 7bcb81b7a81c6872e9549e3a93bbc06c
Sep 26 15:02:21 dist-test-slave-7153 krb5kdc[9716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434941, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.123.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.67@KRBTEST.COM" created.
I20260926 15:02:21.937449 10086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:21.918024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:47191 (local address 127.9.123.126:41577)
0926 15:02:21.918546 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:21.918553 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:21.919623 (+ 1070us) server_negotiation.cc:407] Connection header received
0926 15:02:21.921551 (+ 1928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:21.921555 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:21.921635 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:21.921966 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:21.923608 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:21.924316 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:21.925934 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:21.926378 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:21.926492 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:21.930498 (+ 4006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:21.930504 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:21.930507 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:21.930546 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:21.933973 (+ 3427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:21.934857 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:21.934862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:21.934864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:21.935036 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:21.935511 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:21.935515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:21.935517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:21.935838 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:21.936006 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:21.936782 (+ 776us) server_negotiation.cc:299] Negotiation successful
0926 15:02:21.937031 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":259,"threads_started":1}
I20260926 15:02:21.938320 10087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:21.918684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:41577 (local address 127.9.123.66:47191)
0926 15:02:21.919472 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:21.919518 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:21.921287 (+ 1769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:21.922240 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:21.922249 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:21.922810 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:21.923428 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:21.923447 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:21.924463 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:21.924467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:21.925781 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:21.925794 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:21.926610 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:21.926617 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:21.926855 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:21.927543 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:21.927569 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:21.930178 (+ 2609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:21.934170 (+ 3992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:21.934178 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:21.934195 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:21.934684 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:21.935154 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:21.935158 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:21.935161 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:21.935363 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:21.935994 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:21.936001 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:21.936393 (+ 392us) client_negotiation.cc:769] Sending connection context
0926 15:02:21.936633 (+ 240us) client_negotiation.cc:240] Negotiation successful
0926 15:02:21.937051 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":266,"threads_started":1}
I20260926 15:02:21.939855 10084 heartbeater.cc:347] Connected to a master server at 127.9.123.126:41577
I20260926 15:02:21.940212 10084 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:21.940997 10084 heartbeater.cc:511] Master 127.9.123.126:41577 requested a full tablet report, sending...
I20260926 15:02:21.942549 9761 ts_manager.cc:194] Registered new tserver with Master: 4262af25a9f3425392b3914625a92945 (127.9.123.66:45071)
I20260926 15:02:21.943339 9761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.66@KRBTEST.COM'} at 127.9.123.66:47191
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
I20260926 15:02:21.957461 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:41577
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:35099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:22.293505 10090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:22.293792 10090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:22.293913 10090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:22.301271 10090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:22.301388 10090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:22.301522 10090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:02:22.311834 10090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:35099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:41577
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:22.313194 10090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:22.314451 10090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:22.327576 10098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:22.327639 10096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:22.327633 10095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:22.328703 10090 server_base.cc:1034] running on GCE node
I20260926 15:02:22.329396 10090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:22.330348 10090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:22.331552 10090 hybrid_clock.cc:648] HybridClock initialized: now 1790434942331491 us; error 58 us; skew 500 ppm
I20260926 15:02:22.331863 10090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:22 dist-test-slave-7153 krb5kdc[9716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434942, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:22.335135 10090 init.cc:377] Logged in from keytab as kudu/127.9.123.67@KRBTEST.COM (short username kudu)
I20260926 15:02:22.336710 10090 webserver.cc:460] Webserver started at http://127.9.123.67:34097/ using document root <none> and password file <none>
I20260926 15:02:22.337445 10090 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:22.337548 10090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:22.337815 10090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:22.340818 10090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "0a8b2422585b4caa922946a347d460ce"
format_stamp: "Formatted at 2026-09-26 15:02:22 on dist-test-slave-7153"
server_key: "eef2d0cab5617e0c5d5c8e686868a9b4"
server_key_iv: "f1b6615d23d930d69352bfa84643502f"
server_key_version: "encryptionkey@0"
I20260926 15:02:22.341638 10090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "0a8b2422585b4caa922946a347d460ce"
format_stamp: "Formatted at 2026-09-26 15:02:22 on dist-test-slave-7153"
server_key: "eef2d0cab5617e0c5d5c8e686868a9b4"
server_key_iv: "f1b6615d23d930d69352bfa84643502f"
server_key_version: "encryptionkey@0"
I20260926 15:02:22.347097 10090 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 15:02:22.351357 10105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:22.352936 10090 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260926 15:02:22.353093 10090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "0a8b2422585b4caa922946a347d460ce"
format_stamp: "Formatted at 2026-09-26 15:02:22 on dist-test-slave-7153"
server_key: "eef2d0cab5617e0c5d5c8e686868a9b4"
server_key_iv: "f1b6615d23d930d69352bfa84643502f"
server_key_version: "encryptionkey@0"
I20260926 15:02:22.353367 10090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:22.384251 10090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:22.388576 10090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:22.388836 10090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:22.389823 10090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:02:22.391201 10090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:22.391279 10090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:22.391373 10090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:22.391414 10090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:22.411700 9950 heartbeater.cc:503] Master 127.9.123.126:41577 was elected leader, sending a full tablet report...
I20260926 15:02:22.425621 10090 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:46501
I20260926 15:02:22.425670 10217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:46501 every 8 connection(s)
I20260926 15:02:22.426890 10090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:02:22.434509 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 10090
I20260926 15:02:22.434737 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:02:22.435300 9709 external_mini_cluster.cc:1428] Setting key c4d8fae09f4b54267776a4424242839e
Sep 26 15:02:22 dist-test-slave-7153 krb5kdc[9716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434942, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:22.448989 10086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:22.429512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:43123 (local address 127.9.123.126:41577)
0926 15:02:22.429674 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:22.429679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:22.430903 (+ 1224us) server_negotiation.cc:407] Connection header received
0926 15:02:22.432875 (+ 1972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:22.432881 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:22.432981 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:22.433068 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:22.435014 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:22.436115 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:22.438328 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:22.438752 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:22.438875 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:22.443250 (+ 4375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:22.443255 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:22.443259 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:22.443306 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:22.445975 (+ 2669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:22.446825 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:22.446830 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:22.446832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:22.446957 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:22.447427 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:22.447432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:22.447434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:22.447722 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:22.447855 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:22.448569 (+ 714us) server_negotiation.cc:299] Negotiation successful
0926 15:02:22.448754 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 15:02:22.450011 10220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:22.430022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:41577 (local address 127.9.123.67:43123)
0926 15:02:22.430728 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:22.430773 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:22.432552 (+ 1779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:22.433382 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:22.433392 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:22.433944 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:22.434802 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:22.434830 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:22.436305 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:22.436309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:22.438195 (+ 1886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:22.438209 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:22.439187 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:22.439192 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:22.439451 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:22.440077 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:22.440105 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:22.442938 (+ 2833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:22.446162 (+ 3224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:22.446171 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:22.446187 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:22.446679 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:22.447044 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:22.447047 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:22.447049 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:22.447236 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:22.447827 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:22.447834 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:22.448186 (+ 352us) client_negotiation.cc:769] Sending connection context
0926 15:02:22.448453 (+ 267us) client_negotiation.cc:240] Negotiation successful
0926 15:02:22.448858 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":280,"threads_started":1}
I20260926 15:02:22.451558 10218 heartbeater.cc:347] Connected to a master server at 127.9.123.126:41577
I20260926 15:02:22.451879 10218 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:22.452625 10218 heartbeater.cc:511] Master 127.9.123.126:41577 requested a full tablet report, sending...
I20260926 15:02:22.454003 9761 ts_manager.cc:194] Registered new tserver with Master: 0a8b2422585b4caa922946a347d460ce (127.9.123.67:46501)
I20260926 15:02:22.454788 9761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.67@KRBTEST.COM'} at 127.9.123.67:43123
I20260926 15:02:22.464600 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 15:02:22 dist-test-slave-7153 krb5kdc[9716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434942, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 15:02:22 dist-test-slave-7153 krb5kdc[9716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434942, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.123.65@KRBTEST.COM
I20260926 15:02:22.508167 10225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:22.492477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50916 (local address 127.9.123.65:42425)
0926 15:02:22.493562 (+ 1085us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:22.493569 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:22.493597 (+ 28us) server_negotiation.cc:407] Connection header received
0926 15:02:22.493657 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:22.493663 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:22.493884 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:22.494022 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:22.495447 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:22.496445 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:22.497826 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:22.498461 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:22.498565 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:22.502493 (+ 3928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:22.502508 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:22.502523 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:22.502561 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:22.505396 (+ 2835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:22.506087 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:22.506112 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:22.506114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:22.506257 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:22.506615 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:22.506617 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:22.506620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:22.506970 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:22.507190 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:22.507615 (+ 425us) server_negotiation.cc:299] Negotiation successful
0926 15:02:22.507850 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":940,"thread_start_us":366,"threads_started":1}
W20260926 15:02:22.509507 9865 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50916
Sep 26 15:02:22 dist-test-slave-7153 krb5kdc[9716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 15:02:22 dist-test-slave-7153 krb5kdc[9716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.65@KRBTEST.COM
I20260926 15:02:22.551915 10225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:22.537940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50926 (local address 127.9.123.65:42425)
0926 15:02:22.538174 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:22.538181 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:22.538436 (+ 255us) server_negotiation.cc:407] Connection header received
0926 15:02:22.538611 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:22.538634 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:22.538734 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:22.538826 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:22.540176 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:22.540862 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:22.542139 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:22.542572 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:22.542694 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:22.546636 (+ 3942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:22.546642 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:22.546645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:22.546684 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:22.549346 (+ 2662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:22.550070 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:22.550075 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:22.550077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:22.550267 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:22.550662 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:22.550666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:22.550668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:22.550951 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:22.551178 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:22.551497 (+ 319us) server_negotiation.cc:299] Negotiation successful
0926 15:02:22.551679 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260926 15:02:22.553579 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 9822
I20260926 15:02:22.566834 10225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:22.554650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50930 (local address 127.9.123.65:42425)
0926 15:02:22.554860 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:22.554866 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:22.555075 (+ 209us) server_negotiation.cc:407] Connection header received
0926 15:02:22.555243 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:22.555248 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:22.555310 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:22.555388 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:22.556548 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:22.557373 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:22.558599 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:22.558995 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:22.559100 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:22.561303 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:22.561308 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:22.561311 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:22.561342 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:22.564331 (+ 2989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:22.565000 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:22.565006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:22.565009 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:22.565184 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:22.565510 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:22.565515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:22.565518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:22.565774 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:22.565996 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:22.566286 (+ 290us) server_negotiation.cc:299] Negotiation successful
0926 15:02:22.566499 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 15:02:22.567585 9945 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:02:22.649683 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 9822
I20260926 15:02:22.663262 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 9956
Sep 26 15:02:22 dist-test-slave-7153 krb5kdc[9716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.66@KRBTEST.COM
I20260926 15:02:22.679435 10232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:22.664244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60880 (local address 127.9.123.66:45071)
0926 15:02:22.664778 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:22.664786 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:22.664810 (+ 24us) server_negotiation.cc:407] Connection header received
0926 15:02:22.664867 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:22.664874 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:22.665080 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:22.665241 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:22.666443 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:22.667539 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:22.668781 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:22.669411 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:22.669513 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:22.673616 (+ 4103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:22.673628 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:22.673645 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:22.673682 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:22.676598 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:22.677405 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:22.677415 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:22.677419 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:22.677586 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:22.677961 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:22.677964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:22.677966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:22.678304 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:22.678532 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:22.678842 (+ 310us) server_negotiation.cc:299] Negotiation successful
0926 15:02:22.679038 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":247,"threads_started":1}
I20260926 15:02:22.680773 10079 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:02:22.764559 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 9956
I20260926 15:02:22.782812 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 10090
Sep 26 15:02:22 dist-test-slave-7153 krb5kdc[9716](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.67@KRBTEST.COM
I20260926 15:02:22.799806 10234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:22.783984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34028 (local address 127.9.123.67:46501)
0926 15:02:22.784606 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:22.784615 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:22.784663 (+ 48us) server_negotiation.cc:407] Connection header received
0926 15:02:22.784733 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:22.784740 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:22.785000 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:22.785192 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:22.786451 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:22.787509 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:22.788812 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:22.789460 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:22.789582 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:22.794214 (+ 4632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:22.794230 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:22.794248 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:22.794293 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:22.797089 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:22.798030 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:22.798037 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:22.798039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:22.798165 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:22.798473 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:22.798477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:22.798479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:22.798831 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:22.799062 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:22.799279 (+ 217us) server_negotiation.cc:299] Negotiation successful
0926 15:02:22.799518 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":309,"threads_started":1}
I20260926 15:02:22.801270 10213 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:02:22.884689 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 10090
I20260926 15:02:22.898679 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 9729
I20260926 15:02:22.899981 9791 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:02:23.002892 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 9729
2026-09-26T15:02:23Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3865 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:02:23 dist-test-slave-7153 krb5kdc[10239](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:02:23 dist-test-slave-7153 krb5kdc[10239](info): set up 2 sockets
Sep 26 15:02:23 dist-test-slave-7153 krb5kdc[10239](info): commencing operation
krb5kdc: starting...
W20260926 15:02:24.008935 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.968s user 0.003s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:02:24 dist-test-slave-7153 krb5kdc[10239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:02:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:02:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:02:24.184970 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:46729
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:42599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:46729
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:24.503827 10252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:24.504154 10252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:24.504288 10252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:24.511685 10252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:02:24.511832 10252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:24.511900 10252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:24.511976 10252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:02:24.512034 10252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:02:24.522059 10252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:42599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:46729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:46729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:24.523452 10252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:24.524767 10252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:24.536172 10257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:24.536243 10260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:24.536204 10258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:24.536948 10252 server_base.cc:1034] running on GCE node
I20260926 15:02:24.537783 10252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:24.539220 10252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:24.540445 10252 hybrid_clock.cc:648] HybridClock initialized: now 1790434944540401 us; error 40 us; skew 500 ppm
I20260926 15:02:24.540789 10252 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:24 dist-test-slave-7153 krb5kdc[10239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434944, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:24.543850 10252 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:02:24.545442 10252 webserver.cc:460] Webserver started at http://127.9.123.126:35147/ using document root <none> and password file <none>
I20260926 15:02:24.546181 10252 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:24.546305 10252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:24.546610 10252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:24.549759 10252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "003d69d0ef904249a503ade4659f3f1a"
format_stamp: "Formatted at 2026-09-26 15:02:24 on dist-test-slave-7153"
server_key: "ccc0d16ccc8fb1f6eb7dd17a0a0ea1a6"
server_key_iv: "92869596f86a9aad2e7e31f5d4e77464"
server_key_version: "encryptionkey@0"
I20260926 15:02:24.550623 10252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "003d69d0ef904249a503ade4659f3f1a"
format_stamp: "Formatted at 2026-09-26 15:02:24 on dist-test-slave-7153"
server_key: "ccc0d16ccc8fb1f6eb7dd17a0a0ea1a6"
server_key_iv: "92869596f86a9aad2e7e31f5d4e77464"
server_key_version: "encryptionkey@0"
I20260926 15:02:24.556267 10252 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 15:02:24.559836 10267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:24.561427 10252 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 15:02:24.561609 10252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "003d69d0ef904249a503ade4659f3f1a"
format_stamp: "Formatted at 2026-09-26 15:02:24 on dist-test-slave-7153"
server_key: "ccc0d16ccc8fb1f6eb7dd17a0a0ea1a6"
server_key_iv: "92869596f86a9aad2e7e31f5d4e77464"
server_key_version: "encryptionkey@0"
I20260926 15:02:24.561811 10252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:24.611838 10252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:24.615978 10252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:24.616242 10252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:24.633643 10252 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:46729
I20260926 15:02:24.633651 10318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:46729 every 8 connection(s)
I20260926 15:02:24.635005 10252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:02:24.638751 10319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:24.641878 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 10252
I20260926 15:02:24.642094 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:02:24.642764 9709 external_mini_cluster.cc:1428] Setting key e6eafb46e6a59bdcc157fb5020248b8c
I20260926 15:02:24.652020 10319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 003d69d0ef904249a503ade4659f3f1a: Bootstrap starting.
I20260926 15:02:24.655630 10319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 003d69d0ef904249a503ade4659f3f1a: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:24.656644 10319 log.cc:824] T 00000000000000000000000000000000 P 003d69d0ef904249a503ade4659f3f1a: Log is configured to *not* fsync() on all Append() calls
Sep 26 15:02:24 dist-test-slave-7153 krb5kdc[10239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:24.660003 10319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 003d69d0ef904249a503ade4659f3f1a: No bootstrap required, opened a new log
I20260926 15:02:24.666882 10322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:24.644859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55218 (local address 127.9.123.126:46729)
0926 15:02:24.645652 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:24.645668 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:24.645719 (+ 51us) server_negotiation.cc:407] Connection header received
0926 15:02:24.647698 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:24.647733 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:24.648343 (+ 610us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:24.648831 (+ 488us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:24.650318 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:24.652290 (+ 1972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:24.653696 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:24.654744 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:24.654928 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:24.658985 (+ 4057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:24.658993 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:24.659012 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:24.659053 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:24.662306 (+ 3253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:24.663202 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:24.663209 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:24.663220 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:24.663391 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:24.663766 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:24.663769 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:24.663771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:24.664316 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:24.664538 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:24.665192 (+ 654us) server_negotiation.cc:299] Negotiation successful
0926 15:02:24.665693 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":343,"threads_started":1}
I20260926 15:02:24.668699 10319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 003d69d0ef904249a503ade4659f3f1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003d69d0ef904249a503ade4659f3f1a" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 46729 } }
I20260926 15:02:24.669045 10319 raft_consensus.cc:374] T 00000000000000000000000000000000 P 003d69d0ef904249a503ade4659f3f1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:24.669183 10319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 003d69d0ef904249a503ade4659f3f1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 003d69d0ef904249a503ade4659f3f1a, State: Initialized, Role: FOLLOWER
I20260926 15:02:24.669734 10319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 003d69d0ef904249a503ade4659f3f1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003d69d0ef904249a503ade4659f3f1a" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 46729 } }
I20260926 15:02:24.669960 10319 raft_consensus.cc:388] T 00000000000000000000000000000000 P 003d69d0ef904249a503ade4659f3f1a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:02:24.670078 10319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 003d69d0ef904249a503ade4659f3f1a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:02:24.670257 10319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 003d69d0ef904249a503ade4659f3f1a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:24.672300 10319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 003d69d0ef904249a503ade4659f3f1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003d69d0ef904249a503ade4659f3f1a" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 46729 } }
I20260926 15:02:24.672751 10319 leader_election.cc:302] T 00000000000000000000000000000000 P 003d69d0ef904249a503ade4659f3f1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 003d69d0ef904249a503ade4659f3f1a; no voters:
I20260926 15:02:24.673336 10319 leader_election.cc:288] T 00000000000000000000000000000000 P 003d69d0ef904249a503ade4659f3f1a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:02:24.673478 10324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 003d69d0ef904249a503ade4659f3f1a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:24.674566 10324 raft_consensus.cc:686] T 00000000000000000000000000000000 P 003d69d0ef904249a503ade4659f3f1a [term 1 LEADER]: Becoming Leader. State: Replica: 003d69d0ef904249a503ade4659f3f1a, State: Running, Role: LEADER
I20260926 15:02:24.675083 10324 consensus_queue.cc:237] T 00000000000000000000000000000000 P 003d69d0ef904249a503ade4659f3f1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003d69d0ef904249a503ade4659f3f1a" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 46729 } }
I20260926 15:02:24.675494 10319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 003d69d0ef904249a503ade4659f3f1a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:02:24.677616 10325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 003d69d0ef904249a503ade4659f3f1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "003d69d0ef904249a503ade4659f3f1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003d69d0ef904249a503ade4659f3f1a" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 46729 } } }
I20260926 15:02:24.677640 10326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 003d69d0ef904249a503ade4659f3f1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 003d69d0ef904249a503ade4659f3f1a. Latest consensus state: current_term: 1 leader_uuid: "003d69d0ef904249a503ade4659f3f1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003d69d0ef904249a503ade4659f3f1a" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 46729 } } }
I20260926 15:02:24.678036 10325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 003d69d0ef904249a503ade4659f3f1a [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:24.678036 10326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 003d69d0ef904249a503ade4659f3f1a [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:24.678785 10330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:02:24.687110 10330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:02:24.699385 10330 catalog_manager.cc:1347] Generated new cluster ID: 1d01114a90ca4d1e8b15f01d014bd792
I20260926 15:02:24.699505 10330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:02:24.726421 10330 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:02:24.727795 10330 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:02:24.738312 10330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 003d69d0ef904249a503ade4659f3f1a: Generated new TSK 0
I20260926 15:02:24.739238 10330 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.123.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
I20260926 15:02:24.785562 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:46729
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:42599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:25.111708 10345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:25.112008 10345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:25.112128 10345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:25.119395 10345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:25.119531 10345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:25.119690 10345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:02:25.129604 10345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:42599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:46729
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:25.131078 10345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:25.132759 10345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:25.145233 10351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:25.145314 10350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:25.145663 10353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:25.146384 10345 server_base.cc:1034] running on GCE node
I20260926 15:02:25.147002 10345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:25.147884 10345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:25.149084 10345 hybrid_clock.cc:648] HybridClock initialized: now 1790434945149033 us; error 45 us; skew 500 ppm
I20260926 15:02:25.149432 10345 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:25 dist-test-slave-7153 krb5kdc[10239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434945, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:25.152456 10345 init.cc:377] Logged in from keytab as kudu/127.9.123.65@KRBTEST.COM (short username kudu)
I20260926 15:02:25.154011 10345 webserver.cc:460] Webserver started at http://127.9.123.65:39353/ using document root <none> and password file <none>
I20260926 15:02:25.154690 10345 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:25.154812 10345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:25.155118 10345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:25.158165 10345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "b38515876a1f40769a6d5efa90dd4b53"
format_stamp: "Formatted at 2026-09-26 15:02:25 on dist-test-slave-7153"
server_key: "175e74be9caed6de48b89b17648ab947"
server_key_iv: "2ac772f910de900e336142a2793dfea6"
server_key_version: "encryptionkey@0"
I20260926 15:02:25.158998 10345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "b38515876a1f40769a6d5efa90dd4b53"
format_stamp: "Formatted at 2026-09-26 15:02:25 on dist-test-slave-7153"
server_key: "175e74be9caed6de48b89b17648ab947"
server_key_iv: "2ac772f910de900e336142a2793dfea6"
server_key_version: "encryptionkey@0"
I20260926 15:02:25.164218 10345 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 15:02:25.167790 10360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:25.169402 10345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 15:02:25.169574 10345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "b38515876a1f40769a6d5efa90dd4b53"
format_stamp: "Formatted at 2026-09-26 15:02:25 on dist-test-slave-7153"
server_key: "175e74be9caed6de48b89b17648ab947"
server_key_iv: "2ac772f910de900e336142a2793dfea6"
server_key_version: "encryptionkey@0"
I20260926 15:02:25.169764 10345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:25.190374 10345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:25.194841 10345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:25.195134 10345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:25.196101 10345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:02:25.197569 10345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:25.197647 10345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:25.197765 10345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:25.197815 10345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:25.228193 10345 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:45193
I20260926 15:02:25.228242 10472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:45193 every 8 connection(s)
I20260926 15:02:25.229537 10345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:02:25.232622 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 10345
I20260926 15:02:25.232815 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:02:25.233493 9709 external_mini_cluster.cc:1428] Setting key 3d745e94b684fcf46292b13d4ea0936d
Sep 26 15:02:25 dist-test-slave-7153 krb5kdc[10239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434945, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.123.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.66@KRBTEST.COM" created.
I20260926 15:02:25.250217 10475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:25.232030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:44043 (local address 127.9.123.126:46729)
0926 15:02:25.232498 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:25.232505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:25.233682 (+ 1177us) server_negotiation.cc:407] Connection header received
0926 15:02:25.235548 (+ 1866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:25.235553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:25.235635 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:25.235763 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:25.237439 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:25.238465 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:25.240111 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:25.240584 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:25.240705 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:25.244171 (+ 3466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:25.244176 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:25.244179 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:25.244214 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:25.246805 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:25.247700 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:25.247704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:25.247715 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:25.247858 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:25.248274 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:25.248278 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:25.248281 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:25.248563 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:25.248752 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:25.249500 (+ 748us) server_negotiation.cc:299] Negotiation successful
0926 15:02:25.249809 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":225,"threads_started":1}
I20260926 15:02:25.251170 10476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:25.232661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:46729 (local address 127.9.123.65:44043)
0926 15:02:25.233509 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:25.233556 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:25.235265 (+ 1709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:25.235982 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:25.235990 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:25.236534 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:25.237244 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:25.237274 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:25.238622 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:25.238626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:25.239968 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:25.239981 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:25.240820 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:25.240838 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:25.241060 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:25.241739 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:25.241767 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:25.243928 (+ 2161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:25.247028 (+ 3100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:25.247038 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:25.247059 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:25.247561 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:25.247965 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:25.247970 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:25.247973 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:25.248157 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:25.248691 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:25.248700 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:25.249081 (+ 381us) client_negotiation.cc:769] Sending connection context
0926 15:02:25.249391 (+ 310us) client_negotiation.cc:240] Negotiation successful
0926 15:02:25.249958 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":342,"threads_started":1}
I20260926 15:02:25.252848 10473 heartbeater.cc:347] Connected to a master server at 127.9.123.126:46729
I20260926 15:02:25.253233 10473 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:25.254122 10473 heartbeater.cc:511] Master 127.9.123.126:46729 requested a full tablet report, sending...
I20260926 15:02:25.256565 10284 ts_manager.cc:194] Registered new tserver with Master: b38515876a1f40769a6d5efa90dd4b53 (127.9.123.65:45193)
I20260926 15:02:25.258639 10284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.65@KRBTEST.COM'} at 127.9.123.65:44043
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
I20260926 15:02:25.276628 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:46729
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:42599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:25.596642 10479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:25.596938 10479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:25.597060 10479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:25.604403 10479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:25.604532 10479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:25.604692 10479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:02:25.614615 10479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:42599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:46729
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:25.615931 10479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:25.617307 10479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:25.629794 10487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:25.629765 10485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:25.630141 10484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:25.630786 10479 server_base.cc:1034] running on GCE node
I20260926 15:02:25.631404 10479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:25.632294 10479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:25.633512 10479 hybrid_clock.cc:648] HybridClock initialized: now 1790434945633440 us; error 58 us; skew 500 ppm
I20260926 15:02:25.633888 10479 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:25 dist-test-slave-7153 krb5kdc[10239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434945, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:25.637514 10479 init.cc:377] Logged in from keytab as kudu/127.9.123.66@KRBTEST.COM (short username kudu)
I20260926 15:02:25.639159 10479 webserver.cc:460] Webserver started at http://127.9.123.66:35907/ using document root <none> and password file <none>
I20260926 15:02:25.639995 10479 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:25.640112 10479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:25.640414 10479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:25.644196 10479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "79569bb0ba13434297a30f988bd17f99"
format_stamp: "Formatted at 2026-09-26 15:02:25 on dist-test-slave-7153"
server_key: "f7604734ea6a21dc9cb3db4e7b6398b8"
server_key_iv: "94a7d3fc286af6dc0c4d862cf00d9f32"
server_key_version: "encryptionkey@0"
I20260926 15:02:25.645259 10479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "79569bb0ba13434297a30f988bd17f99"
format_stamp: "Formatted at 2026-09-26 15:02:25 on dist-test-slave-7153"
server_key: "f7604734ea6a21dc9cb3db4e7b6398b8"
server_key_iv: "94a7d3fc286af6dc0c4d862cf00d9f32"
server_key_version: "encryptionkey@0"
I20260926 15:02:25.652307 10479 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260926 15:02:25.656243 10494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:25.657653 10479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:02:25.657791 10479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "79569bb0ba13434297a30f988bd17f99"
format_stamp: "Formatted at 2026-09-26 15:02:25 on dist-test-slave-7153"
server_key: "f7604734ea6a21dc9cb3db4e7b6398b8"
server_key_iv: "94a7d3fc286af6dc0c4d862cf00d9f32"
server_key_version: "encryptionkey@0"
I20260926 15:02:25.657991 10479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:25.675215 10479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:25.679701 10479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:25.679999 10479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:25.680979 10479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:02:25.682343 10479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:25.682415 10479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:25.682504 10479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:25.682546 10479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:25.712370 10479 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:40081
I20260926 15:02:25.712466 10606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:40081 every 8 connection(s)
I20260926 15:02:25.713645 10479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:02:25.722761 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 10479
I20260926 15:02:25.722955 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:02:25.723501 9709 external_mini_cluster.cc:1428] Setting key dd4a6d1ec0400bf6b699f1645149b292
Sep 26 15:02:25 dist-test-slave-7153 krb5kdc[10239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434945, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:25.734503 10475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:25.716078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:46395 (local address 127.9.123.126:46729)
0926 15:02:25.716242 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:25.716248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:25.717433 (+ 1185us) server_negotiation.cc:407] Connection header received
0926 15:02:25.719143 (+ 1710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:25.719147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:25.719213 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:25.719295 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:25.720726 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:25.721487 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:25.722998 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:25.723589 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:25.723711 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:25.727160 (+ 3449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:25.727167 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:25.727172 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:25.727210 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:25.730134 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:25.731064 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:25.731069 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:25.731072 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:25.731203 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:25.731642 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:25.731645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:25.731648 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:25.731896 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:25.731999 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:25.732821 (+ 822us) server_negotiation.cc:299] Negotiation successful
0926 15:02:25.733060 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 15:02:25.734834 10609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:25.716589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:46729 (local address 127.9.123.66:46395)
0926 15:02:25.717283 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:25.717328 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:25.718903 (+ 1575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:25.719489 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:25.719497 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:25.720020 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:25.720587 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:25.720605 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:25.721613 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:25.721617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:25.722871 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:25.722883 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:25.723801 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:25.723807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:25.723997 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:25.724680 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:25.724707 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:25.726925 (+ 2218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:25.730271 (+ 3346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:25.730281 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:25.730301 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:25.730920 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:25.731310 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:25.731315 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:25.731318 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:25.731548 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:25.732004 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:25.732013 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:25.732462 (+ 449us) client_negotiation.cc:769] Sending connection context
0926 15:02:25.732777 (+ 315us) client_negotiation.cc:240] Negotiation successful
0926 15:02:25.733440 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":247,"threads_started":1}
I20260926 15:02:25.736352 10607 heartbeater.cc:347] Connected to a master server at 127.9.123.126:46729
I20260926 15:02:25.736706 10607 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:25.737684 10607 heartbeater.cc:511] Master 127.9.123.126:46729 requested a full tablet report, sending...
I20260926 15:02:25.739411 10284 ts_manager.cc:194] Registered new tserver with Master: 79569bb0ba13434297a30f988bd17f99 (127.9.123.66:40081)
WARNING: no policy specified for kudu/127.9.123.67@KRBTEST.COM; defaulting to no policy
I20260926 15:02:25.740532 10284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.66@KRBTEST.COM'} at 127.9.123.66:46395
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
I20260926 15:02:25.763928 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:46729
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:42599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:26.079097 10612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:26.079388 10612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:26.079510 10612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:26.086947 10612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:26.087085 10612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:26.087232 10612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:02:26.097210 10612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:42599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:46729
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:26.098543 10612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:26.100080 10612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:26.112986 10618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:26.113009 10620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:26.113284 10617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:26.113842 10612 server_base.cc:1034] running on GCE node
I20260926 15:02:26.114472 10612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:26.115413 10612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:26.116616 10612 hybrid_clock.cc:648] HybridClock initialized: now 1790434946116562 us; error 43 us; skew 500 ppm
I20260926 15:02:26.116925 10612 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:26 dist-test-slave-7153 krb5kdc[10239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434946, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:26.119992 10612 init.cc:377] Logged in from keytab as kudu/127.9.123.67@KRBTEST.COM (short username kudu)
I20260926 15:02:26.121582 10612 webserver.cc:460] Webserver started at http://127.9.123.67:37619/ using document root <none> and password file <none>
I20260926 15:02:26.122259 10612 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:26.122380 10612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:26.122676 10612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:26.125608 10612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "d74b068143c044ce975ea318175fd32c"
format_stamp: "Formatted at 2026-09-26 15:02:26 on dist-test-slave-7153"
server_key: "dcf0c882e1c949bdcef25c2b958156c2"
server_key_iv: "c44fa1adfdd40ff766d6d37bc5a63ef8"
server_key_version: "encryptionkey@0"
I20260926 15:02:26.126398 10612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "d74b068143c044ce975ea318175fd32c"
format_stamp: "Formatted at 2026-09-26 15:02:26 on dist-test-slave-7153"
server_key: "dcf0c882e1c949bdcef25c2b958156c2"
server_key_iv: "c44fa1adfdd40ff766d6d37bc5a63ef8"
server_key_version: "encryptionkey@0"
I20260926 15:02:26.131712 10612 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 15:02:26.135413 10627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:26.136822 10612 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 15:02:26.136986 10612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "d74b068143c044ce975ea318175fd32c"
format_stamp: "Formatted at 2026-09-26 15:02:26 on dist-test-slave-7153"
server_key: "dcf0c882e1c949bdcef25c2b958156c2"
server_key_iv: "c44fa1adfdd40ff766d6d37bc5a63ef8"
server_key_version: "encryptionkey@0"
I20260926 15:02:26.137238 10612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:26.172062 10612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:26.176645 10612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:26.176918 10612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:26.177978 10612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:02:26.179339 10612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:26.179415 10612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:26.179504 10612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:26.179545 10612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:26.211390 10612 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:38621
I20260926 15:02:26.211434 10739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:38621 every 8 connection(s)
I20260926 15:02:26.212685 10612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:02:26.220372 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 10612
I20260926 15:02:26.220567 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:02:26.220969 9709 external_mini_cluster.cc:1428] Setting key f6dae2a8cbe36397e4d87601bfab7ce8
Sep 26 15:02:26 dist-test-slave-7153 krb5kdc[10239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434946, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:26.233456 10475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:26.215294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:47227 (local address 127.9.123.126:46729)
0926 15:02:26.215474 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:26.215479 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:26.216721 (+ 1242us) server_negotiation.cc:407] Connection header received
0926 15:02:26.218558 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:26.218564 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:26.218648 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:26.218740 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:26.220473 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.221342 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.223029 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.223711 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.223836 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:26.227528 (+ 3692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:26.227534 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:26.227537 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:26.227574 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:26.230147 (+ 2573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:26.231125 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:26.231129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:26.231132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:26.231274 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:26.231759 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:26.231762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:26.231765 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:26.232041 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:26.232152 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:26.232980 (+ 828us) server_negotiation.cc:299] Negotiation successful
0926 15:02:26.233198 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 15:02:26.234746 10742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:26.215792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:46729 (local address 127.9.123.67:47227)
0926 15:02:26.216525 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:26.216570 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:26.218274 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:26.218992 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:26.219002 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:26.219554 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:26.220259 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:26.220286 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.221496 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.221500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:26.222913 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:26.222924 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.223926 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.223934 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:26.224174 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:26.224819 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:26.224864 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:26.227259 (+ 2395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:26.230322 (+ 3063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:26.230334 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:26.230352 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:26.230989 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:26.231381 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:26.231386 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:26.231390 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:26.231633 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:26.232165 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:26.232173 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:26.232602 (+ 429us) client_negotiation.cc:769] Sending connection context
0926 15:02:26.232908 (+ 306us) client_negotiation.cc:240] Negotiation successful
0926 15:02:26.233415 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":281,"threads_started":1}
I20260926 15:02:26.236244 10740 heartbeater.cc:347] Connected to a master server at 127.9.123.126:46729
I20260926 15:02:26.236585 10740 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:26.237386 10740 heartbeater.cc:511] Master 127.9.123.126:46729 requested a full tablet report, sending...
I20260926 15:02:26.238835 10284 ts_manager.cc:194] Registered new tserver with Master: d74b068143c044ce975ea318175fd32c (127.9.123.67:38621)
I20260926 15:02:26.239627 10284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.67@KRBTEST.COM'} at 127.9.123.67:47227
I20260926 15:02:26.249547 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 15:02:26.263170 10473 heartbeater.cc:503] Master 127.9.123.126:46729 was elected leader, sending a full tablet report...
I20260926 15:02:26.269176 10475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:26.256984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55234 (local address 127.9.123.126:46729)
0926 15:02:26.257191 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:26.257198 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:26.257605 (+ 407us) server_negotiation.cc:407] Connection header received
0926 15:02:26.257776 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:26.257780 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:26.257851 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:26.257935 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:26.259338 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.260036 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.261343 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.261773 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.261887 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:26.264228 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:26.264233 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:26.264236 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:26.264295 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:26.266902 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:26.267543 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:26.267546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:26.267548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:26.267675 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:26.267955 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:26.267957 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:26.267959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:26.268250 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:26.268448 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:26.268751 (+ 303us) server_negotiation.cc:299] Negotiation successful
0926 15:02:26.268918 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 15:02:26.276077 10284 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:02:26.283996 10284 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:02:26.307315 10755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:26.297189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42354 (local address 127.9.123.66:40081)
0926 15:02:26.298643 (+ 1454us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:26.298651 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:26.298679 (+ 28us) server_negotiation.cc:407] Connection header received
0926 15:02:26.298795 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:26.298803 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:26.299060 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:26.300668 (+ 1608us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:26.300889 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.302270 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.304665 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.305909 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.306039 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:26.306276 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:26.306835 (+ 559us) server_negotiation.cc:299] Negotiation successful
0926 15:02:26.306975 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1327,"thread_start_us":1220,"threads_started":1}
I20260926 15:02:26.307523 10754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:26.296548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:40081 (local address 127.0.0.1:42354)
0926 15:02:26.297913 (+ 1365us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:26.297964 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:26.298142 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:26.300000 (+ 1858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:26.300008 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:26.300039 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:26.300488 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:26.300502 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.302431 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.302436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:26.304517 (+ 2081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:26.304529 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.306400 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.306407 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:26.306597 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:26.306640 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 15:02:26.306756 (+ 116us) client_negotiation.cc:240] Negotiation successful
0926 15:02:26.307205 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1122,"thread_start_us":2649,"threads_started":1}
I20260926 15:02:26.308995 10757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:26.297761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46298 (local address 127.9.123.65:45193)
0926 15:02:26.300057 (+ 2296us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:26.300066 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:26.300716 (+ 650us) server_negotiation.cc:407] Connection header received
0926 15:02:26.300814 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:26.300821 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:26.301073 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:26.301288 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:26.302180 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.303893 (+ 1713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.306457 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.307944 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.308078 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:26.308328 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:26.308534 (+ 206us) server_negotiation.cc:299] Negotiation successful
0926 15:02:26.308700 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2138,"thread_start_us":2012,"threads_started":1}
I20260926 15:02:26.310325 10752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:26.296162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:38621 (local address 127.0.0.1:53344)
0926 15:02:26.298684 (+ 2522us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:26.298700 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:26.298807 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:26.301318 (+ 2511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:26.301322 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:26.301356 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:26.301994 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:26.302004 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.303821 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.303827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:26.306153 (+ 2326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:26.306185 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.309572 (+ 3387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.309576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:26.309694 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:26.309715 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 15:02:26.309802 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 15:02:26.309933 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"mutex_wait_us":70,"spinlock_wait_cycles":2944,"thread_start_us":598,"threads_started":1}
I20260926 15:02:26.310317 10753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:26.296400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53344 (local address 127.9.123.67:38621)
0926 15:02:26.300830 (+ 4430us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:26.300837 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:26.300862 (+ 25us) server_negotiation.cc:407] Connection header received
0926 15:02:26.300936 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:26.300941 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:26.301139 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:26.302152 (+ 1013us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:26.302428 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.303664 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.306384 (+ 2720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.309205 (+ 2821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.309319 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:26.309471 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:26.309861 (+ 390us) server_negotiation.cc:299] Negotiation successful
0926 15:02:26.310017 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4263,"thread_start_us":4130,"threads_started":1}
I20260926 15:02:26.310680 10756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:26.297308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:45193 (local address 127.0.0.1:46298)
0926 15:02:26.300488 (+ 3180us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:26.300505 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:26.300617 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:26.301296 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:26.301301 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:26.301335 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:26.301788 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:26.301800 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.304034 (+ 2234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.304038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:26.306277 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:26.306297 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.308143 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.308150 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:26.308328 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:26.308360 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 15:02:26.308444 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 15:02:26.308571 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3052,"mutex_wait_us":20,"thread_start_us":2904,"threads_started":1}
I20260926 15:02:26.311617 10542 tablet_service.cc:1467] Processing CreateTablet for tablet d106bf3ec73440c9a7420a92ce71bd98 (DEFAULT_TABLE table=test-table [id=90d4e7328aeb471e9dad4ace98c32a0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:26.313201 10542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d106bf3ec73440c9a7420a92ce71bd98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:26.314311 10408 tablet_service.cc:1467] Processing CreateTablet for tablet d106bf3ec73440c9a7420a92ce71bd98 (DEFAULT_TABLE table=test-table [id=90d4e7328aeb471e9dad4ace98c32a0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:26.314471 10675 tablet_service.cc:1467] Processing CreateTablet for tablet d106bf3ec73440c9a7420a92ce71bd98 (DEFAULT_TABLE table=test-table [id=90d4e7328aeb471e9dad4ace98c32a0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:26.315678 10408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d106bf3ec73440c9a7420a92ce71bd98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:26.316256 10675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d106bf3ec73440c9a7420a92ce71bd98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:26.326927 10758 tablet_bootstrap.cc:492] T d106bf3ec73440c9a7420a92ce71bd98 P b38515876a1f40769a6d5efa90dd4b53: Bootstrap starting.
I20260926 15:02:26.329037 10759 tablet_bootstrap.cc:492] T d106bf3ec73440c9a7420a92ce71bd98 P 79569bb0ba13434297a30f988bd17f99: Bootstrap starting.
I20260926 15:02:26.330842 10760 tablet_bootstrap.cc:492] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c: Bootstrap starting.
I20260926 15:02:26.331692 10758 tablet_bootstrap.cc:654] T d106bf3ec73440c9a7420a92ce71bd98 P b38515876a1f40769a6d5efa90dd4b53: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:26.332513 10759 tablet_bootstrap.cc:654] T d106bf3ec73440c9a7420a92ce71bd98 P 79569bb0ba13434297a30f988bd17f99: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:26.333148 10758 log.cc:824] T d106bf3ec73440c9a7420a92ce71bd98 P b38515876a1f40769a6d5efa90dd4b53: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:26.333938 10759 log.cc:824] T d106bf3ec73440c9a7420a92ce71bd98 P 79569bb0ba13434297a30f988bd17f99: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:26.334755 10760 tablet_bootstrap.cc:654] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:26.335816 10760 log.cc:824] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:26.336042 10758 tablet_bootstrap.cc:492] T d106bf3ec73440c9a7420a92ce71bd98 P b38515876a1f40769a6d5efa90dd4b53: No bootstrap required, opened a new log
I20260926 15:02:26.336289 10758 ts_tablet_manager.cc:1397] T d106bf3ec73440c9a7420a92ce71bd98 P b38515876a1f40769a6d5efa90dd4b53: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260926 15:02:26.336779 10759 tablet_bootstrap.cc:492] T d106bf3ec73440c9a7420a92ce71bd98 P 79569bb0ba13434297a30f988bd17f99: No bootstrap required, opened a new log
I20260926 15:02:26.337018 10759 ts_tablet_manager.cc:1397] T d106bf3ec73440c9a7420a92ce71bd98 P 79569bb0ba13434297a30f988bd17f99: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260926 15:02:26.338706 10760 tablet_bootstrap.cc:492] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c: No bootstrap required, opened a new log
I20260926 15:02:26.339006 10760 ts_tablet_manager.cc:1397] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260926 15:02:26.344781 10758 raft_consensus.cc:348] T d106bf3ec73440c9a7420a92ce71bd98 P b38515876a1f40769a6d5efa90dd4b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74b068143c044ce975ea318175fd32c" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 38621 } } peers { permanent_uuid: "79569bb0ba13434297a30f988bd17f99" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 40081 } } peers { permanent_uuid: "b38515876a1f40769a6d5efa90dd4b53" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45193 } }
I20260926 15:02:26.345176 10758 raft_consensus.cc:374] T d106bf3ec73440c9a7420a92ce71bd98 P b38515876a1f40769a6d5efa90dd4b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:26.345301 10758 raft_consensus.cc:729] T d106bf3ec73440c9a7420a92ce71bd98 P b38515876a1f40769a6d5efa90dd4b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38515876a1f40769a6d5efa90dd4b53, State: Initialized, Role: FOLLOWER
I20260926 15:02:26.345899 10758 consensus_queue.cc:260] T d106bf3ec73440c9a7420a92ce71bd98 P b38515876a1f40769a6d5efa90dd4b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74b068143c044ce975ea318175fd32c" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 38621 } } peers { permanent_uuid: "79569bb0ba13434297a30f988bd17f99" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 40081 } } peers { permanent_uuid: "b38515876a1f40769a6d5efa90dd4b53" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45193 } }
I20260926 15:02:26.347357 10759 raft_consensus.cc:348] T d106bf3ec73440c9a7420a92ce71bd98 P 79569bb0ba13434297a30f988bd17f99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74b068143c044ce975ea318175fd32c" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 38621 } } peers { permanent_uuid: "79569bb0ba13434297a30f988bd17f99" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 40081 } } peers { permanent_uuid: "b38515876a1f40769a6d5efa90dd4b53" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45193 } }
I20260926 15:02:26.347913 10759 raft_consensus.cc:374] T d106bf3ec73440c9a7420a92ce71bd98 P 79569bb0ba13434297a30f988bd17f99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:26.348045 10759 raft_consensus.cc:729] T d106bf3ec73440c9a7420a92ce71bd98 P 79569bb0ba13434297a30f988bd17f99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79569bb0ba13434297a30f988bd17f99, State: Initialized, Role: FOLLOWER
I20260926 15:02:26.348199 10758 ts_tablet_manager.cc:1428] T d106bf3ec73440c9a7420a92ce71bd98 P b38515876a1f40769a6d5efa90dd4b53: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260926 15:02:26.348685 10759 consensus_queue.cc:260] T d106bf3ec73440c9a7420a92ce71bd98 P 79569bb0ba13434297a30f988bd17f99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74b068143c044ce975ea318175fd32c" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 38621 } } peers { permanent_uuid: "79569bb0ba13434297a30f988bd17f99" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 40081 } } peers { permanent_uuid: "b38515876a1f40769a6d5efa90dd4b53" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45193 } }
I20260926 15:02:26.349781 10607 heartbeater.cc:503] Master 127.9.123.126:46729 was elected leader, sending a full tablet report...
I20260926 15:02:26.350435 10759 ts_tablet_manager.cc:1428] T d106bf3ec73440c9a7420a92ce71bd98 P 79569bb0ba13434297a30f988bd17f99: Time spent starting tablet: real 0.013s user 0.007s sys 0.005s
I20260926 15:02:26.351902 10760 raft_consensus.cc:348] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74b068143c044ce975ea318175fd32c" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 38621 } } peers { permanent_uuid: "79569bb0ba13434297a30f988bd17f99" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 40081 } } peers { permanent_uuid: "b38515876a1f40769a6d5efa90dd4b53" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45193 } }
I20260926 15:02:26.352284 10760 raft_consensus.cc:374] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:26.352393 10760 raft_consensus.cc:729] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74b068143c044ce975ea318175fd32c, State: Initialized, Role: FOLLOWER
I20260926 15:02:26.352982 10760 consensus_queue.cc:260] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74b068143c044ce975ea318175fd32c" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 38621 } } peers { permanent_uuid: "79569bb0ba13434297a30f988bd17f99" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 40081 } } peers { permanent_uuid: "b38515876a1f40769a6d5efa90dd4b53" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45193 } }
I20260926 15:02:26.354099 10740 heartbeater.cc:503] Master 127.9.123.126:46729 was elected leader, sending a full tablet report...
I20260926 15:02:26.354617 10760 ts_tablet_manager.cc:1428] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c: Time spent starting tablet: real 0.016s user 0.013s sys 0.000s
W20260926 15:02:26.464334 10741 tablet.cc:2369] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 15:02:26.465504 10608 tablet.cc:2369] T d106bf3ec73440c9a7420a92ce71bd98 P 79569bb0ba13434297a30f988bd17f99: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 15:02:26.481822 10474 tablet.cc:2369] T d106bf3ec73440c9a7420a92ce71bd98 P b38515876a1f40769a6d5efa90dd4b53: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:02:26.638813 10766 raft_consensus.cc:482] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:26.639102 10766 raft_consensus.cc:504] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74b068143c044ce975ea318175fd32c" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 38621 } } peers { permanent_uuid: "79569bb0ba13434297a30f988bd17f99" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 40081 } } peers { permanent_uuid: "b38515876a1f40769a6d5efa90dd4b53" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45193 } }
I20260926 15:02:26.640722 10766 leader_election.cc:288] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79569bb0ba13434297a30f988bd17f99 (127.9.123.66:40081), b38515876a1f40769a6d5efa90dd4b53 (127.9.123.65:45193)
I20260926 15:02:26.647781 10757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:26.641312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:33539 (local address 127.9.123.65:45193)
0926 15:02:26.641511 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:26.641519 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:26.641843 (+ 324us) server_negotiation.cc:407] Connection header received
0926 15:02:26.641918 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:26.641923 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:26.641994 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:26.642062 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:26.642573 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.643465 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.645459 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.646753 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.646914 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:26.646989 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:26.647397 (+ 408us) server_negotiation.cc:299] Negotiation successful
0926 15:02:26.647535 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 15:02:26.647819 10742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:26.640991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:45193 (local address 127.9.123.67:33539)
0926 15:02:26.641576 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:26.641597 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:26.641729 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:26.642210 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:26.642215 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:26.642238 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:26.642425 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:26.642434 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.643580 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.643586 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:26.645302 (+ 1716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:26.645317 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.646921 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.646928 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:26.647181 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:26.647240 (+ 59us) client_negotiation.cc:769] Sending connection context
0926 15:02:26.647373 (+ 133us) client_negotiation.cc:240] Negotiation successful
0926 15:02:26.647517 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465}
I20260926 15:02:26.648572 10755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:26.641681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:59403 (local address 127.9.123.66:40081)
0926 15:02:26.641862 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:26.641867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:26.641885 (+ 18us) server_negotiation.cc:407] Connection header received
0926 15:02:26.642116 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:26.642122 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:26.642180 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:26.642256 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:26.643024 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.644086 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.646138 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.647513 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.647657 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:26.647737 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:26.648214 (+ 477us) server_negotiation.cc:299] Negotiation successful
0926 15:02:26.648343 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 15:02:26.648633 10767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:26.641125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:40081 (local address 127.9.123.67:59403)
0926 15:02:26.641815 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:26.641832 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:26.641948 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:26.642419 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:26.642422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:26.642440 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:26.642840 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:26.642852 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.644247 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.644253 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:26.645983 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:26.645993 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.647879 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.647885 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:26.648048 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:26.648078 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 15:02:26.648158 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 15:02:26.648286 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"mutex_wait_us":48,"thread_start_us":785,"threads_started":1}
I20260926 15:02:26.648934 10428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d106bf3ec73440c9a7420a92ce71bd98" candidate_uuid: "d74b068143c044ce975ea318175fd32c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38515876a1f40769a6d5efa90dd4b53" is_pre_election: true
I20260926 15:02:26.649533 10428 raft_consensus.cc:2454] T d106bf3ec73440c9a7420a92ce71bd98 P b38515876a1f40769a6d5efa90dd4b53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74b068143c044ce975ea318175fd32c in term 0.
I20260926 15:02:26.649567 10562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d106bf3ec73440c9a7420a92ce71bd98" candidate_uuid: "d74b068143c044ce975ea318175fd32c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79569bb0ba13434297a30f988bd17f99" is_pre_election: true
I20260926 15:02:26.650074 10562 raft_consensus.cc:2454] T d106bf3ec73440c9a7420a92ce71bd98 P 79569bb0ba13434297a30f988bd17f99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74b068143c044ce975ea318175fd32c in term 0.
I20260926 15:02:26.650295 10630 leader_election.cc:302] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b38515876a1f40769a6d5efa90dd4b53, d74b068143c044ce975ea318175fd32c; no voters:
I20260926 15:02:26.650732 10766 raft_consensus.cc:2781] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:26.650882 10766 raft_consensus.cc:482] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:26.650974 10766 raft_consensus.cc:3037] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:26.653117 10766 raft_consensus.cc:504] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74b068143c044ce975ea318175fd32c" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 38621 } } peers { permanent_uuid: "79569bb0ba13434297a30f988bd17f99" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 40081 } } peers { permanent_uuid: "b38515876a1f40769a6d5efa90dd4b53" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45193 } }
I20260926 15:02:26.653697 10766 leader_election.cc:288] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c [CANDIDATE]: Term 1 election: Requested vote from peers 79569bb0ba13434297a30f988bd17f99 (127.9.123.66:40081), b38515876a1f40769a6d5efa90dd4b53 (127.9.123.65:45193)
I20260926 15:02:26.654201 10562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d106bf3ec73440c9a7420a92ce71bd98" candidate_uuid: "d74b068143c044ce975ea318175fd32c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79569bb0ba13434297a30f988bd17f99"
I20260926 15:02:26.654279 10428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d106bf3ec73440c9a7420a92ce71bd98" candidate_uuid: "d74b068143c044ce975ea318175fd32c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b38515876a1f40769a6d5efa90dd4b53"
I20260926 15:02:26.654428 10562 raft_consensus.cc:3037] T d106bf3ec73440c9a7420a92ce71bd98 P 79569bb0ba13434297a30f988bd17f99 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:26.654461 10428 raft_consensus.cc:3037] T d106bf3ec73440c9a7420a92ce71bd98 P b38515876a1f40769a6d5efa90dd4b53 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:26.657528 10562 raft_consensus.cc:2454] T d106bf3ec73440c9a7420a92ce71bd98 P 79569bb0ba13434297a30f988bd17f99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74b068143c044ce975ea318175fd32c in term 1.
I20260926 15:02:26.657528 10428 raft_consensus.cc:2454] T d106bf3ec73440c9a7420a92ce71bd98 P b38515876a1f40769a6d5efa90dd4b53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74b068143c044ce975ea318175fd32c in term 1.
I20260926 15:02:26.658005 10630 leader_election.cc:302] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b38515876a1f40769a6d5efa90dd4b53, d74b068143c044ce975ea318175fd32c; no voters:
I20260926 15:02:26.658346 10766 raft_consensus.cc:2781] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:26.658775 10766 raft_consensus.cc:686] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c [term 1 LEADER]: Becoming Leader. State: Replica: d74b068143c044ce975ea318175fd32c, State: Running, Role: LEADER
I20260926 15:02:26.659245 10766 consensus_queue.cc:237] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74b068143c044ce975ea318175fd32c" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 38621 } } peers { permanent_uuid: "79569bb0ba13434297a30f988bd17f99" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 40081 } } peers { permanent_uuid: "b38515876a1f40769a6d5efa90dd4b53" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45193 } }
I20260926 15:02:26.663647 10283 catalog_manager.cc:5495] T d106bf3ec73440c9a7420a92ce71bd98 P d74b068143c044ce975ea318175fd32c reported cstate change: term changed from 0 to 1, leader changed from <none> to d74b068143c044ce975ea318175fd32c (127.9.123.67). New cstate: current_term: 1 leader_uuid: "d74b068143c044ce975ea318175fd32c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d74b068143c044ce975ea318175fd32c" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 38621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79569bb0ba13434297a30f988bd17f99" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 40081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b38515876a1f40769a6d5efa90dd4b53" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45193 } health_report { overall_health: UNKNOWN } } }
Sep 26 15:02:26 dist-test-slave-7153 krb5kdc[10239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.65@KRBTEST.COM
I20260926 15:02:26.771057 10757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:26.755948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46304 (local address 127.9.123.65:45193)
0926 15:02:26.756168 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:26.756175 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:26.756646 (+ 471us) server_negotiation.cc:407] Connection header received
0926 15:02:26.756825 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:26.756831 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:26.756907 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:26.756992 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:26.758476 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.759200 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.760451 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.760899 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.761008 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:26.765303 (+ 4295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:26.765320 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:26.765337 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:26.765377 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:26.768282 (+ 2905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:26.768966 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:26.768974 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:26.768977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:26.769150 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:26.769564 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:26.769569 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:26.769573 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:26.769936 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:26.770207 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:26.770512 (+ 305us) server_negotiation.cc:299] Negotiation successful
0926 15:02:26.770721 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
Sep 26 15:02:26 dist-test-slave-7153 krb5kdc[10239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434946, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 15:02:26 dist-test-slave-7153 krb5kdc[10239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434946, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.123.65@KRBTEST.COM
I20260926 15:02:26.814049 10757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:26.799833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46314 (local address 127.9.123.65:45193)
0926 15:02:26.800052 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:26.800058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:26.800373 (+ 315us) server_negotiation.cc:407] Connection header received
0926 15:02:26.800542 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:26.800546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:26.800608 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:26.800691 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:26.802064 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.802814 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.804059 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.804470 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.804578 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:26.808941 (+ 4363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:26.808947 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:26.808950 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:26.808997 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:26.811502 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:26.812138 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:26.812143 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:26.812145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:26.812308 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:26.812687 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:26.812697 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:26.812699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:26.812964 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:26.813262 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:26.813530 (+ 268us) server_negotiation.cc:299] Negotiation successful
0926 15:02:26.813747 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
Sep 26 15:02:26 dist-test-slave-7153 krb5kdc[10239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 15:02:26 dist-test-slave-7153 krb5kdc[10239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.65@KRBTEST.COM
I20260926 15:02:26.856474 10757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:26.843156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46316 (local address 127.9.123.65:45193)
0926 15:02:26.843413 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:26.843418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:26.843530 (+ 112us) server_negotiation.cc:407] Connection header received
0926 15:02:26.843676 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:26.843680 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:26.843745 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:26.843830 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:26.845156 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.845845 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.847052 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.847457 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.847547 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:26.851326 (+ 3779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:26.851332 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:26.851336 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:26.851397 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:26.853814 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:26.854509 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:26.854513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:26.854515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:26.854658 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:26.855005 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:26.855008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:26.855010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:26.855289 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:26.855477 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:26.855937 (+ 460us) server_negotiation.cc:299] Negotiation successful
0926 15:02:26.856111 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260926 15:02:26.858088 10388 tablet_service.cc:2908] Rejecting scan request for tablet d106bf3ec73440c9a7420a92ce71bd98: Uninitialized: clean time has not yet been initialized
I20260926 15:02:26.861141 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 10345
I20260926 15:02:26.873073 10757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:26.861945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46330 (local address 127.9.123.65:45193)
0926 15:02:26.862125 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:26.862130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:26.862400 (+ 270us) server_negotiation.cc:407] Connection header received
0926 15:02:26.862546 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:26.862549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:26.862620 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:26.862690 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:26.863782 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.864450 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.865717 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.866113 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.866313 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:26.868362 (+ 2049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:26.868367 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:26.868370 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:26.868410 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:26.870848 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:26.871433 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:26.871437 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:26.871440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:26.871557 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:26.871892 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:26.871897 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:26.871900 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:26.872248 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:26.872430 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:26.872635 (+ 205us) server_negotiation.cc:299] Negotiation successful
0926 15:02:26.872796 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 15:02:26.873746 10468 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:02:26.962075 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 10345
I20260926 15:02:26.975297 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 10479
Sep 26 15:02:26 dist-test-slave-7153 krb5kdc[10239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.66@KRBTEST.COM
I20260926 15:02:26.989490 10755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:26.976059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42366 (local address 127.9.123.66:40081)
0926 15:02:26.976229 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:26.976233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:26.976251 (+ 18us) server_negotiation.cc:407] Connection header received
0926 15:02:26.976379 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:26.976383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:26.976444 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:26.976531 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:26.977792 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.978545 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.979821 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:26.980256 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:26.980364 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:26.984155 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:26.984171 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:26.984187 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:26.984225 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:26.986894 (+ 2669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:26.987554 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:26.987563 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:26.987565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:26.987700 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:26.987993 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:26.987996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:26.987998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:26.988347 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:26.988567 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:26.988903 (+ 336us) server_negotiation.cc:299] Negotiation successful
0926 15:02:26.989073 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 15:02:26.990124 10602 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:02:27.077514 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 10479
I20260926 15:02:27.090276 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 10612
Sep 26 15:02:27 dist-test-slave-7153 krb5kdc[10239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.67@KRBTEST.COM
I20260926 15:02:27.105077 10783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:27.091114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53350 (local address 127.9.123.67:38621)
0926 15:02:27.091573 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:27.091578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:27.091593 (+ 15us) server_negotiation.cc:407] Connection header received
0926 15:02:27.091646 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:27.091650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:27.091716 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:27.091812 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:27.093042 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:27.093815 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:27.095114 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:27.095529 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:27.095649 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:27.099685 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:27.099701 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:27.099717 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:27.099755 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:27.102385 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:27.103095 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:27.103103 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:27.103105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:27.103243 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:27.103611 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:27.103615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:27.103617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:27.103957 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:27.104232 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:27.104498 (+ 266us) server_negotiation.cc:299] Negotiation successful
0926 15:02:27.104718 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":246,"threads_started":1}
I20260926 15:02:27.105850 10735 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:02:27.192730 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 10612
I20260926 15:02:27.205560 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 10252
I20260926 15:02:27.206712 10314 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:02:27.312407 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 10252
2026-09-26T15:02:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4310 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:02:27 dist-test-slave-7153 krb5kdc[10789](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:02:27 dist-test-slave-7153 krb5kdc[10789](info): set up 2 sockets
Sep 26 15:02:27 dist-test-slave-7153 krb5kdc[10789](info): commencing operation
krb5kdc: starting...
W20260926 15:02:28.321015 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.970s user 0.002s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:02:28 dist-test-slave-7153 krb5kdc[10789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:02:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:02:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:02:28.500504 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:41757
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:36195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:41757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:28.820526 10802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:28.820849 10802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:28.820981 10802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:28.828503 10802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:02:28.828619 10802 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 15:02:28.828682 10802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:28.828768 10802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:28.828830 10802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:02:28.828919 10802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:02:28.839206 10802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:36195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:41757
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:41757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:28.840567 10802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:28.841952 10802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:28.854171 10807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:28.854143 10808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:28.854204 10810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:28.855132 10802 server_base.cc:1034] running on GCE node
I20260926 15:02:28.856009 10802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:28.857617 10802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:28.858843 10802 hybrid_clock.cc:648] HybridClock initialized: now 1790434948858778 us; error 51 us; skew 500 ppm
I20260926 15:02:28.859192 10802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:28 dist-test-slave-7153 krb5kdc[10789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434948, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:28.869481 10802 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:02:28.871299 10802 webserver.cc:460] Webserver started at http://127.9.123.126:34017/ using document root <none> and password file <none>
I20260926 15:02:28.872087 10802 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:28.872216 10802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:28.872562 10802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:28.875773 10802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "4effaa816ce84ebd9e2b0f7dd6f383f1"
format_stamp: "Formatted at 2026-09-26 15:02:28 on dist-test-slave-7153"
server_key: "a5ac8f56f25a77ac7706565b9b9b4e51"
server_key_iv: "35064ac3cd36e33f7b30f254252096e9"
server_key_version: "encryptionkey@0"
I20260926 15:02:28.876636 10802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "4effaa816ce84ebd9e2b0f7dd6f383f1"
format_stamp: "Formatted at 2026-09-26 15:02:28 on dist-test-slave-7153"
server_key: "a5ac8f56f25a77ac7706565b9b9b4e51"
server_key_iv: "35064ac3cd36e33f7b30f254252096e9"
server_key_version: "encryptionkey@0"
I20260926 15:02:28.882319 10802 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 15:02:28.886933 10817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:28.889120 10802 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260926 15:02:28.889365 10802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "4effaa816ce84ebd9e2b0f7dd6f383f1"
format_stamp: "Formatted at 2026-09-26 15:02:28 on dist-test-slave-7153"
server_key: "a5ac8f56f25a77ac7706565b9b9b4e51"
server_key_iv: "35064ac3cd36e33f7b30f254252096e9"
server_key_version: "encryptionkey@0"
I20260926 15:02:28.889669 10802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:28.926409 10802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:28.930886 10802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:28.931123 10802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:28.953091 10802 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:41757
I20260926 15:02:28.953137 10878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:41757 every 8 connection(s)
I20260926 15:02:28.954631 10802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:02:28.958101 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 10802
I20260926 15:02:28.958352 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:02:28.959259 9709 external_mini_cluster.cc:1428] Setting key 8f86a57cd8705d865d2c7c71b1b1647b
I20260926 15:02:28.959513 10879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:28.975483 10879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1: Bootstrap starting.
Sep 26 15:02:28 dist-test-slave-7153 krb5kdc[10789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:28.979334 10879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:28.980582 10879 log.cc:824] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:28.984385 10879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1: No bootstrap required, opened a new log
I20260926 15:02:28.986732 10882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:28.961552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55748 (local address 127.9.123.126:41757)
0926 15:02:28.962586 (+ 1034us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:28.962603 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:28.962649 (+ 46us) server_negotiation.cc:407] Connection header received
0926 15:02:28.965511 (+ 2862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:28.965545 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:28.966053 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:28.966486 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:28.968504 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:28.969966 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:28.971844 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:28.972719 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:28.972895 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:28.977933 (+ 5038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:28.977943 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:28.977960 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:28.978011 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:28.981865 (+ 3854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:28.982934 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:28.982941 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:28.982949 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:28.983146 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:28.983733 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:28.983737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:28.983740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:28.984343 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:28.984626 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:28.985170 (+ 544us) server_negotiation.cc:299] Negotiation successful
0926 15:02:28.985596 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":799,"thread_start_us":541,"threads_started":1}
I20260926 15:02:28.993548 10879 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4effaa816ce84ebd9e2b0f7dd6f383f1" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 41757 } }
I20260926 15:02:28.993954 10879 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:28.994066 10879 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4effaa816ce84ebd9e2b0f7dd6f383f1, State: Initialized, Role: FOLLOWER
I20260926 15:02:28.994700 10879 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4effaa816ce84ebd9e2b0f7dd6f383f1" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 41757 } }
I20260926 15:02:28.994947 10879 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:02:28.995076 10879 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:02:28.995240 10879 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:28.998088 10879 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4effaa816ce84ebd9e2b0f7dd6f383f1" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 41757 } }
I20260926 15:02:28.998879 10879 leader_election.cc:302] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4effaa816ce84ebd9e2b0f7dd6f383f1; no voters:
I20260926 15:02:28.999665 10879 leader_election.cc:288] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:02:28.999804 10884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:29.000496 10884 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1 [term 1 LEADER]: Becoming Leader. State: Replica: 4effaa816ce84ebd9e2b0f7dd6f383f1, State: Running, Role: LEADER
I20260926 15:02:29.001165 10884 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4effaa816ce84ebd9e2b0f7dd6f383f1" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 41757 } }
I20260926 15:02:29.001982 10879 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:02:29.005156 10885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4effaa816ce84ebd9e2b0f7dd6f383f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4effaa816ce84ebd9e2b0f7dd6f383f1" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 41757 } } }
I20260926 15:02:29.005527 10885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1 [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:29.004895 10886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4effaa816ce84ebd9e2b0f7dd6f383f1. Latest consensus state: current_term: 1 leader_uuid: "4effaa816ce84ebd9e2b0f7dd6f383f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4effaa816ce84ebd9e2b0f7dd6f383f1" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 41757 } } }
I20260926 15:02:29.006278 10886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1 [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:29.007035 10891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:02:29.017946 10891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:02:29.030663 10891 catalog_manager.cc:1347] Generated new cluster ID: f4d7e12e83914ee6970dffd44b015532
I20260926 15:02:29.030797 10891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:02:29.062718 10891 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:02:29.064076 10891 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:02:29.079555 10891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1: Generated new TSK 0
I20260926 15:02:29.080643 10891 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.123.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
I20260926 15:02:29.143121 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:41757
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:36195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:29.476796 10905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:29.477160 10905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:29.477300 10905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:29.484707 10905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:29.484835 10905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:29.484952 10905 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 15:02:29.485061 10905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:02:29.495311 10905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:36195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.123.126:41757
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:29.496601 10905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:29.498014 10905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:29.511752 10911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:29.511912 10910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:29.512275 10913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:29.512568 10905 server_base.cc:1034] running on GCE node
I20260926 15:02:29.513437 10905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:29.514496 10905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:29.515717 10905 hybrid_clock.cc:648] HybridClock initialized: now 1790434949515641 us; error 57 us; skew 500 ppm
I20260926 15:02:29.516036 10905 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:29 dist-test-slave-7153 krb5kdc[10789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434949, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:29.519258 10905 init.cc:377] Logged in from keytab as kudu/127.9.123.65@KRBTEST.COM (short username kudu)
I20260926 15:02:29.520792 10905 webserver.cc:460] Webserver started at http://127.9.123.65:41563/ using document root <none> and password file <none>
I20260926 15:02:29.521569 10905 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:29.521677 10905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:29.521997 10905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:29.525048 10905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "18694435286247378305d101ea5ddee8"
format_stamp: "Formatted at 2026-09-26 15:02:29 on dist-test-slave-7153"
server_key: "0af6606ee6cc6648cd9f975bdd9023f7"
server_key_iv: "bdc1842bcb77014780aa531b3b379cf4"
server_key_version: "encryptionkey@0"
I20260926 15:02:29.526037 10905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "18694435286247378305d101ea5ddee8"
format_stamp: "Formatted at 2026-09-26 15:02:29 on dist-test-slave-7153"
server_key: "0af6606ee6cc6648cd9f975bdd9023f7"
server_key_iv: "bdc1842bcb77014780aa531b3b379cf4"
server_key_version: "encryptionkey@0"
I20260926 15:02:29.532181 10905 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.002s
I20260926 15:02:29.536082 10920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:29.537633 10905 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:02:29.537797 10905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "18694435286247378305d101ea5ddee8"
format_stamp: "Formatted at 2026-09-26 15:02:29 on dist-test-slave-7153"
server_key: "0af6606ee6cc6648cd9f975bdd9023f7"
server_key_iv: "bdc1842bcb77014780aa531b3b379cf4"
server_key_version: "encryptionkey@0"
I20260926 15:02:29.538009 10905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:29.565066 10905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:29.569392 10905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:29.569665 10905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:29.572471 10905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:29.572578 10905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:29.572722 10905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:29.572779 10905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 15:02:29 dist-test-slave-7153 krb5kdc[10789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434949, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:29.596242 10929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:29.573238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:34225 (local address 127.9.123.126:41757)
0926 15:02:29.573672 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:29.573678 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:29.575186 (+ 1508us) server_negotiation.cc:407] Connection header received
0926 15:02:29.578110 (+ 2924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:29.578117 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:29.578216 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:29.578359 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:29.581073 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:29.582051 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:29.584078 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:29.584504 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:29.584620 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:29.589807 (+ 5187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:29.589813 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:29.589816 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:29.589859 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:29.592377 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:29.593497 (+ 1120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:29.593504 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:29.593508 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:29.593683 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:29.594250 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:29.594256 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:29.594259 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:29.594651 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:29.594794 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:29.595636 (+ 842us) server_negotiation.cc:299] Negotiation successful
0926 15:02:29.595885 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":203,"threads_started":1}
I20260926 15:02:29.597795 10930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:29.573965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:41757 (local address 127.9.123.65:34225)
0926 15:02:29.574937 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:29.574999 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:29.577727 (+ 2728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:29.578712 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:29.578727 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:29.579698 (+ 971us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:29.580835 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:29.580866 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:29.582272 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:29.582278 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:29.583906 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:29.583922 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:29.584748 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:29.584756 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:29.584993 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:29.585913 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:29.585951 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:29.589503 (+ 3552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:29.592537 (+ 3034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:29.592549 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:29.592575 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:29.593341 (+ 766us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:29.593813 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:29.593818 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:29.593821 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:29.594110 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:29.594798 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:29.594808 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:29.595231 (+ 423us) client_negotiation.cc:769] Sending connection context
0926 15:02:29.595497 (+ 266us) client_negotiation.cc:240] Negotiation successful
0926 15:02:29.596113 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":395,"threads_started":1}
I20260926 15:02:29.618918 10929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:29.604626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:48337 (local address 127.9.123.126:41757)
0926 15:02:29.604834 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:29.604841 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:29.605703 (+ 862us) server_negotiation.cc:407] Connection header received
0926 15:02:29.605836 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:29.605842 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:29.605932 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:29.606031 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:29.607522 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:29.608212 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:29.609828 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:29.610486 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:29.610620 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:29.612905 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:29.612911 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:29.612914 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:29.612946 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:29.615833 (+ 2887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:29.616663 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:29.616667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:29.616670 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:29.616823 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:29.617281 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:29.617286 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:29.617289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:29.617666 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:29.617816 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:29.618325 (+ 509us) server_negotiation.cc:299] Negotiation successful
0926 15:02:29.618533 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 15:02:29.619014 10994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:29.604628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:41757 (local address 127.9.123.65:48337)
0926 15:02:29.605594 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:29.605613 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:29.605739 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:29.606082 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:29.606085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:29.606884 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:29.607346 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:29.607359 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:29.608354 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:29.608359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:29.609677 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:29.609696 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:29.610696 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:29.610703 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:29.610901 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:29.611565 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:29.611589 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:29.612689 (+ 1100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:29.615964 (+ 3275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:29.615969 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:29.615973 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:29.616533 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:29.616941 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:29.616949 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:29.616953 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:29.617155 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:29.617807 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:29.617813 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:29.618039 (+ 226us) client_negotiation.cc:769] Sending connection context
0926 15:02:29.618192 (+ 153us) client_negotiation.cc:240] Negotiation successful
0926 15:02:29.618555 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":813,"thread_start_us":609,"threads_started":1}
I20260926 15:02:29.625624 10905 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:46371
I20260926 15:02:29.625669 11040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:46371 every 8 connection(s)
I20260926 15:02:29.626818 10905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:02:29.628746 11041 heartbeater.cc:347] Connected to a master server at 127.9.123.126:41757
I20260926 15:02:29.628954 11041 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:29.629627 11041 heartbeater.cc:511] Master 127.9.123.126:41757 requested a full tablet report, sending...
I20260926 15:02:29.631488 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 10905
I20260926 15:02:29.631549 10834 ts_manager.cc:194] Registered new tserver with Master: 18694435286247378305d101ea5ddee8 (127.9.123.65:46371)
I20260926 15:02:29.631671 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:02:29.632257 9709 external_mini_cluster.cc:1428] Setting key 20dc4a44cce64c62e7b5bd71f7ba09dd
I20260926 15:02:29.633056 10834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.65@KRBTEST.COM'} at 127.9.123.65:34225
WARNING: no policy specified for kudu/127.9.123.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
I20260926 15:02:29.669754 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:41757
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:36195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:29.991860 11045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:29.992162 11045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:29.992293 11045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:29.999820 11045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:29.999961 11045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:30.000090 11045 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 15:02:30.000200 11045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:02:30.010805 11045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:36195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.123.126:41757
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:30.012167 11045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:30.013581 11045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:30.027020 11050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:30.028448 11051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:30.028652 11053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:30.035465 11045 server_base.cc:1034] running on GCE node
I20260926 15:02:30.036223 11045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:30.037358 11045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:30.038616 11045 hybrid_clock.cc:648] HybridClock initialized: now 1790434950038530 us; error 52 us; skew 500 ppm
I20260926 15:02:30.038936 11045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:30 dist-test-slave-7153 krb5kdc[10789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434950, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:30.042207 11045 init.cc:377] Logged in from keytab as kudu/127.9.123.66@KRBTEST.COM (short username kudu)
I20260926 15:02:30.043834 11045 webserver.cc:460] Webserver started at http://127.9.123.66:41011/ using document root <none> and password file <none>
I20260926 15:02:30.044639 11045 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:30.044740 11045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:30.045058 11045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:30.048086 11045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "4f66bf759ff54616807d15a79d4d7a88"
format_stamp: "Formatted at 2026-09-26 15:02:30 on dist-test-slave-7153"
server_key: "bfd98c517367d00c43f8bbffdbd525ec"
server_key_iv: "6657c8013dc985a729bb1559dc8c1b06"
server_key_version: "encryptionkey@0"
I20260926 15:02:30.048882 11045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "4f66bf759ff54616807d15a79d4d7a88"
format_stamp: "Formatted at 2026-09-26 15:02:30 on dist-test-slave-7153"
server_key: "bfd98c517367d00c43f8bbffdbd525ec"
server_key_iv: "6657c8013dc985a729bb1559dc8c1b06"
server_key_version: "encryptionkey@0"
I20260926 15:02:30.054304 11045 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 15:02:30.058126 11060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:30.059587 11045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 15:02:30.059749 11045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "4f66bf759ff54616807d15a79d4d7a88"
format_stamp: "Formatted at 2026-09-26 15:02:30 on dist-test-slave-7153"
server_key: "bfd98c517367d00c43f8bbffdbd525ec"
server_key_iv: "6657c8013dc985a729bb1559dc8c1b06"
server_key_version: "encryptionkey@0"
I20260926 15:02:30.059964 11045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:30.076689 11045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:30.081241 11045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:30.081511 11045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:30.084218 11045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:30.084311 11045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:30.084457 11045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:30.084515 11045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 15:02:30 dist-test-slave-7153 krb5kdc[10789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434950, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:30.107939 10929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:30.084847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:43525 (local address 127.9.123.126:41757)
0926 15:02:30.085062 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:30.085068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:30.086807 (+ 1739us) server_negotiation.cc:407] Connection header received
0926 15:02:30.089860 (+ 3053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:30.089867 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:30.089965 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:30.090082 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:30.092065 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.092780 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.095187 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.095780 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.095861 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:30.101036 (+ 5175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:30.101044 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:30.101049 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:30.101093 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:30.104292 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:30.105427 (+ 1135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:30.105433 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:30.105437 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:30.105601 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:30.106135 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:30.106140 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:30.106144 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:30.106444 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:30.106574 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:30.107361 (+ 787us) server_negotiation.cc:299] Negotiation successful
0926 15:02:30.107594 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 15:02:30.109005 11069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:30.085653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:41757 (local address 127.9.123.66:43525)
0926 15:02:30.086579 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:30.086642 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:30.089408 (+ 2766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:30.090407 (+ 999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:30.090421 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:30.091120 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:30.091860 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:30.091879 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.092933 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.092947 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:30.094990 (+ 2043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:30.095004 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.096202 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.096211 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:30.096540 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:30.097368 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:30.097391 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:30.100761 (+ 3370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:30.104452 (+ 3691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:30.104464 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:30.104489 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:30.105267 (+ 778us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:30.105755 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:30.105758 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:30.105761 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:30.106009 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:30.106570 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:30.106579 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:30.106988 (+ 409us) client_negotiation.cc:769] Sending connection context
0926 15:02:30.107246 (+ 258us) client_negotiation.cc:240] Negotiation successful
0926 15:02:30.107796 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":331,"threads_started":1}
I20260926 15:02:30.128417 11138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:30.114694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:41757 (local address 127.9.123.66:38979)
0926 15:02:30.115225 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:30.115240 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:30.115379 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:30.115683 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:30.115688 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:30.116141 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:30.116420 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:30.116430 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.117537 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.117543 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:30.119107 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:30.119128 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.120239 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.120248 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:30.120449 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:30.121190 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:30.121215 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:30.122410 (+ 1195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:30.125367 (+ 2957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:30.125372 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:30.125377 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:30.125904 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:30.126342 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:30.126347 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:30.126351 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:30.126528 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:30.127195 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:30.127200 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:30.127380 (+ 180us) client_negotiation.cc:769] Sending connection context
0926 15:02:30.127522 (+ 142us) client_negotiation.cc:240] Negotiation successful
0926 15:02:30.127863 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":294,"threads_started":1}
I20260926 15:02:30.128564 10929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:30.114563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:38979 (local address 127.9.123.126:41757)
0926 15:02:30.114715 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:30.114720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:30.115331 (+ 611us) server_negotiation.cc:407] Connection header received
0926 15:02:30.115493 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:30.115499 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:30.115567 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:30.115646 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:30.116581 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.117417 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.119454 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.120018 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.120156 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:30.122597 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:30.122602 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:30.122605 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:30.122637 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:30.125199 (+ 2562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:30.126062 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:30.126066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:30.126068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:30.126196 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:30.126631 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:30.126636 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:30.126639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:30.127083 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:30.127222 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:30.128058 (+ 836us) server_negotiation.cc:299] Negotiation successful
0926 15:02:30.128275 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 15:02:30.132972 11045 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:33611
I20260926 15:02:30.132993 11179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:33611 every 8 connection(s)
I20260926 15:02:30.134195 11045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:02:30.136139 11180 heartbeater.cc:347] Connected to a master server at 127.9.123.126:41757
I20260926 15:02:30.136319 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 11045
I20260926 15:02:30.136370 11180 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:30.136525 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:02:30.137027 9709 external_mini_cluster.cc:1428] Setting key 95f3a67b594dfa2669d291d5f1ff0fc6
I20260926 15:02:30.137190 11180 heartbeater.cc:511] Master 127.9.123.126:41757 requested a full tablet report, sending...
I20260926 15:02:30.138605 10834 ts_manager.cc:194] Registered new tserver with Master: 4f66bf759ff54616807d15a79d4d7a88 (127.9.123.66:33611)
I20260926 15:02:30.139555 10834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.66@KRBTEST.COM'} at 127.9.123.66:43525
WARNING: no policy specified for kudu/127.9.123.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
I20260926 15:02:30.173605 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:41757
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:36195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:30.494292 11184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:30.494597 11184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:30.494720 11184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:30.502179 11184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:30.502316 11184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:30.502434 11184 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 15:02:30.502543 11184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:02:30.512693 11184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:36195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.123.126:41757
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:30.514007 11184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:30.515766 11184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:30.529181 11189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:30.529237 11190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:30.529203 11192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:30.530345 11184 server_base.cc:1034] running on GCE node
I20260926 15:02:30.530941 11184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:30.531838 11184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:30.533073 11184 hybrid_clock.cc:648] HybridClock initialized: now 1790434950532993 us; error 51 us; skew 500 ppm
I20260926 15:02:30.533439 11184 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:30 dist-test-slave-7153 krb5kdc[10789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434950, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:30.536501 11184 init.cc:377] Logged in from keytab as kudu/127.9.123.67@KRBTEST.COM (short username kudu)
I20260926 15:02:30.538053 11184 webserver.cc:460] Webserver started at http://127.9.123.67:33201/ using document root <none> and password file <none>
I20260926 15:02:30.538774 11184 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:30.538878 11184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:30.539184 11184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:30.542169 11184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "d212a963434f4673add46800c0a1ea02"
format_stamp: "Formatted at 2026-09-26 15:02:30 on dist-test-slave-7153"
server_key: "53f38b60278575dfcaefeafe6b958f18"
server_key_iv: "63625dd4bc768e402dab1117b90481f9"
server_key_version: "encryptionkey@0"
I20260926 15:02:30.542925 11184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "d212a963434f4673add46800c0a1ea02"
format_stamp: "Formatted at 2026-09-26 15:02:30 on dist-test-slave-7153"
server_key: "53f38b60278575dfcaefeafe6b958f18"
server_key_iv: "63625dd4bc768e402dab1117b90481f9"
server_key_version: "encryptionkey@0"
I20260926 15:02:30.548112 11184 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 15:02:30.551616 11199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:30.552932 11184 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 15:02:30.553067 11184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "d212a963434f4673add46800c0a1ea02"
format_stamp: "Formatted at 2026-09-26 15:02:30 on dist-test-slave-7153"
server_key: "53f38b60278575dfcaefeafe6b958f18"
server_key_iv: "63625dd4bc768e402dab1117b90481f9"
server_key_version: "encryptionkey@0"
I20260926 15:02:30.553295 11184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:30.579527 11184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:30.583819 11184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:30.584074 11184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:30.586845 11184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:30.586942 11184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:30.587081 11184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:30.587142 11184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 15:02:30 dist-test-slave-7153 krb5kdc[10789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434950, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:30.611043 10929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:30.587411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:33679 (local address 127.9.123.126:41757)
0926 15:02:30.587601 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:30.587606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:30.589161 (+ 1555us) server_negotiation.cc:407] Connection header received
0926 15:02:30.592023 (+ 2862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:30.592030 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:30.592121 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:30.592229 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:30.594604 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.595718 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.598646 (+ 2928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.599078 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.599183 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:30.603981 (+ 4798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:30.603986 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:30.603989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:30.604023 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:30.607097 (+ 3074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:30.608125 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:30.608131 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:30.608135 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:30.608301 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:30.608985 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:30.608988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:30.608990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:30.609314 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:30.609452 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:30.610438 (+ 986us) server_negotiation.cc:299] Negotiation successful
0926 15:02:30.610679 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 15:02:30.612061 11208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:30.588024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:41757 (local address 127.9.123.67:33679)
0926 15:02:30.588953 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:30.589008 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:30.591622 (+ 2614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:30.592560 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:30.592573 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:30.593420 (+ 847us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:30.594351 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:30.594375 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.595908 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.595915 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:30.598466 (+ 2551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:30.598485 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.599344 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.599352 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:30.599697 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:30.600580 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:30.600613 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:30.603738 (+ 3125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:30.607310 (+ 3572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:30.607323 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:30.607344 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:30.607999 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:30.608436 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:30.608443 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:30.608447 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:30.608861 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:30.609468 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:30.609481 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:30.610048 (+ 567us) client_negotiation.cc:769] Sending connection context
0926 15:02:30.610289 (+ 241us) client_negotiation.cc:240] Negotiation successful
0926 15:02:30.610877 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":351,"threads_started":1}
I20260926 15:02:30.633035 10929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:30.617962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:58913 (local address 127.9.123.126:41757)
0926 15:02:30.618121 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:30.618127 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:30.618795 (+ 668us) server_negotiation.cc:407] Connection header received
0926 15:02:30.618978 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:30.618984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:30.619052 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:30.619144 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:30.620634 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.621750 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.623898 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.624281 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.624373 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:30.626643 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:30.626649 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:30.626654 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:30.626691 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:30.629961 (+ 3270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:30.630791 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:30.630797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:30.630800 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:30.630937 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:30.631339 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:30.631344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:30.631347 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:30.631728 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:30.631887 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:30.632463 (+ 576us) server_negotiation.cc:299] Negotiation successful
0926 15:02:30.632692 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 15:02:30.633728 11274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:30.618085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:41757 (local address 127.9.123.67:58913)
0926 15:02:30.618703 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:30.618721 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:30.618846 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:30.619186 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:30.619191 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:30.619857 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:30.620438 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:30.620452 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.621964 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.621969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:30.623759 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:30.623775 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.624457 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.624464 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:30.624630 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:30.625340 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:30.625363 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:30.626438 (+ 1075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:30.630115 (+ 3677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:30.630119 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:30.630123 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:30.630662 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:30.631049 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:30.631055 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:30.631060 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:30.631228 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:30.631866 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:30.631875 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:30.632152 (+ 277us) client_negotiation.cc:769] Sending connection context
0926 15:02:30.632329 (+ 177us) client_negotiation.cc:240] Negotiation successful
0926 15:02:30.632733 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":316,"threads_started":1}
I20260926 15:02:30.637075 11041 heartbeater.cc:503] Master 127.9.123.126:41757 was elected leader, sending a full tablet report...
I20260926 15:02:30.638723 11184 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:41459
I20260926 15:02:30.638763 11318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:41459 every 8 connection(s)
I20260926 15:02:30.639860 11184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:02:30.640617 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 11184
I20260926 15:02:30.640810 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:02:30.641284 9709 external_mini_cluster.cc:1428] Setting key 79d9a14a0daf5ff5e0c5c0d441bfa532
I20260926 15:02:30.641880 11319 heartbeater.cc:347] Connected to a master server at 127.9.123.126:41757
I20260926 15:02:30.642095 11319 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:30.642740 11319 heartbeater.cc:511] Master 127.9.123.126:41757 requested a full tablet report, sending...
I20260926 15:02:30.644100 10834 ts_manager.cc:194] Registered new tserver with Master: d212a963434f4673add46800c0a1ea02 (127.9.123.67:41459)
I20260926 15:02:30.644852 10834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.67@KRBTEST.COM'} at 127.9.123.67:33679
I20260926 15:02:30.656728 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 15:02:30 dist-test-slave-7153 krb5kdc[10789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434950, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 15:02:30 dist-test-slave-7153 krb5kdc[10789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434950, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:30.699149 10929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:30.685776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60706 (local address 127.9.123.126:41757)
0926 15:02:30.685996 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:30.686003 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:30.686212 (+ 209us) server_negotiation.cc:407] Connection header received
0926 15:02:30.686457 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:30.686462 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:30.686528 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:30.686612 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:30.687926 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.688643 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.689907 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.690326 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.690437 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:30.694247 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:30.694253 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:30.694256 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:30.694296 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:30.696822 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:30.697455 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:30.697459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:30.697461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:30.697583 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:30.697922 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:30.697927 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:30.697929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:30.698218 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:30.698411 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:30.698630 (+ 219us) server_negotiation.cc:299] Negotiation successful
0926 15:02:30.698828 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 15:02:30.703632 10834 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:02:30.711138 10834 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:02:30.744976 11333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:30.723702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48902 (local address 127.9.123.66:33611)
0926 15:02:30.736880 (+ 13178us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:30.736892 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:30.736929 (+ 37us) server_negotiation.cc:407] Connection header received
0926 15:02:30.737049 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:30.737057 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:30.737406 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:30.737647 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:30.738672 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.740090 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.742697 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.743754 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.743840 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:30.744001 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:30.744506 (+ 505us) server_negotiation.cc:299] Negotiation successful
0926 15:02:30.744671 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13020,"thread_start_us":12880,"threads_started":1}
I20260926 15:02:30.745033 11332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:30.723195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:33611 (local address 127.0.0.1:48902)
0926 15:02:30.733678 (+ 10483us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:30.733751 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:30.734008 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:30.737907 (+ 3899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:30.737915 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:30.737950 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:30.738449 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:30.738467 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.740290 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.740295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:30.742518 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:30.742530 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.744141 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.744145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:30.744283 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:30.744316 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 15:02:30.744427 (+ 111us) client_negotiation.cc:240] Negotiation successful
0926 15:02:30.744630 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10093,"mutex_wait_us":59,"thread_start_us":11058,"threads_started":1}
I20260926 15:02:30.747480 11336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:30.735596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57050 (local address 127.9.123.65:46371)
0926 15:02:30.737223 (+ 1627us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:30.737231 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:30.739213 (+ 1982us) server_negotiation.cc:407] Connection header received
0926 15:02:30.739480 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:30.739488 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:30.739713 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:30.739878 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:30.740760 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.742342 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.744733 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.746194 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.746330 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:30.746542 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:30.747088 (+ 546us) server_negotiation.cc:299] Negotiation successful
0926 15:02:30.747223 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1456,"thread_start_us":1306,"threads_started":1}
I20260926 15:02:30.747496 11337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:30.737317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:46371 (local address 127.0.0.1:57050)
0926 15:02:30.739097 (+ 1780us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:30.739120 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:30.739311 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:30.740095 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:30.740099 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:30.740123 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:30.740579 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:30.740589 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.742505 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.742510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:30.744555 (+ 2045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:30.744568 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.746731 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.746736 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:30.746898 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:30.746927 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 15:02:30.747041 (+ 114us) client_negotiation.cc:240] Negotiation successful
0926 15:02:30.747169 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1650,"mutex_wait_us":30,"thread_start_us":1439,"threads_started":1}
I20260926 15:02:30.749181 11334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:30.723758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:41459 (local address 127.0.0.1:48174)
0926 15:02:30.735995 (+ 12237us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:30.736026 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:30.736179 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:30.738331 (+ 2152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:30.738341 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:30.738377 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:30.738881 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:30.738892 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.740580 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.740585 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:30.743224 (+ 2639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:30.743237 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.745429 (+ 2192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.745437 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:30.745619 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:30.745653 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 15:02:30.745719 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 15:02:30.745842 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12081,"thread_start_us":12533,"threads_started":1}
I20260926 15:02:30.748920 11110 tablet_service.cc:1467] Processing CreateTablet for tablet 0464f198418b485cb9eb6801b481b005 (DEFAULT_TABLE table=test-table [id=510f254776a5474489121f5fceb35d57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:30.750254 11335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:30.724158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48174 (local address 127.9.123.67:41459)
0926 15:02:30.737730 (+ 13572us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:30.737739 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:30.737783 (+ 44us) server_negotiation.cc:407] Connection header received
0926 15:02:30.737890 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:30.737897 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:30.738128 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:30.738338 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:30.739136 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.740438 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.743413 (+ 2975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.744866 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.749443 (+ 4577us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:30.749687 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:30.749780 (+ 93us) server_negotiation.cc:299] Negotiation successful
0926 15:02:30.749948 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13415,"thread_start_us":13253,"threads_started":1}
I20260926 15:02:30.750348 11110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0464f198418b485cb9eb6801b481b005. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:30.750942 10971 tablet_service.cc:1467] Processing CreateTablet for tablet 0464f198418b485cb9eb6801b481b005 (DEFAULT_TABLE table=test-table [id=510f254776a5474489121f5fceb35d57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:30.752396 10971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0464f198418b485cb9eb6801b481b005. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:30.754079 11249 tablet_service.cc:1467] Processing CreateTablet for tablet 0464f198418b485cb9eb6801b481b005 (DEFAULT_TABLE table=test-table [id=510f254776a5474489121f5fceb35d57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:30.755491 11249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0464f198418b485cb9eb6801b481b005. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:30.762614 11338 tablet_bootstrap.cc:492] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88: Bootstrap starting.
I20260926 15:02:30.765630 11338 tablet_bootstrap.cc:654] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:30.766666 11339 tablet_bootstrap.cc:492] T 0464f198418b485cb9eb6801b481b005 P 18694435286247378305d101ea5ddee8: Bootstrap starting.
I20260926 15:02:30.766860 11338 log.cc:824] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:30.769318 11341 tablet_bootstrap.cc:492] T 0464f198418b485cb9eb6801b481b005 P d212a963434f4673add46800c0a1ea02: Bootstrap starting.
I20260926 15:02:30.771552 11339 tablet_bootstrap.cc:654] T 0464f198418b485cb9eb6801b481b005 P 18694435286247378305d101ea5ddee8: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:30.772174 11341 tablet_bootstrap.cc:654] T 0464f198418b485cb9eb6801b481b005 P d212a963434f4673add46800c0a1ea02: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:30.772905 11339 log.cc:824] T 0464f198418b485cb9eb6801b481b005 P 18694435286247378305d101ea5ddee8: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:30.773137 11341 log.cc:824] T 0464f198418b485cb9eb6801b481b005 P d212a963434f4673add46800c0a1ea02: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:30.775655 11341 tablet_bootstrap.cc:492] T 0464f198418b485cb9eb6801b481b005 P d212a963434f4673add46800c0a1ea02: No bootstrap required, opened a new log
I20260926 15:02:30.775828 11338 tablet_bootstrap.cc:492] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88: No bootstrap required, opened a new log
I20260926 15:02:30.775827 11339 tablet_bootstrap.cc:492] T 0464f198418b485cb9eb6801b481b005 P 18694435286247378305d101ea5ddee8: No bootstrap required, opened a new log
I20260926 15:02:30.775919 11341 ts_tablet_manager.cc:1397] T 0464f198418b485cb9eb6801b481b005 P d212a963434f4673add46800c0a1ea02: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260926 15:02:30.776098 11338 ts_tablet_manager.cc:1397] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.002s
I20260926 15:02:30.776096 11339 ts_tablet_manager.cc:1397] T 0464f198418b485cb9eb6801b481b005 P 18694435286247378305d101ea5ddee8: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260926 15:02:30.784164 11341 raft_consensus.cc:348] T 0464f198418b485cb9eb6801b481b005 P d212a963434f4673add46800c0a1ea02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:30.784581 11341 raft_consensus.cc:374] T 0464f198418b485cb9eb6801b481b005 P d212a963434f4673add46800c0a1ea02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:30.784672 11341 raft_consensus.cc:729] T 0464f198418b485cb9eb6801b481b005 P d212a963434f4673add46800c0a1ea02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d212a963434f4673add46800c0a1ea02, State: Initialized, Role: FOLLOWER
I20260926 15:02:30.785324 11341 consensus_queue.cc:260] T 0464f198418b485cb9eb6801b481b005 P d212a963434f4673add46800c0a1ea02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:30.786384 11319 heartbeater.cc:503] Master 127.9.123.126:41757 was elected leader, sending a full tablet report...
I20260926 15:02:30.786608 11339 raft_consensus.cc:348] T 0464f198418b485cb9eb6801b481b005 P 18694435286247378305d101ea5ddee8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:30.787048 11339 raft_consensus.cc:374] T 0464f198418b485cb9eb6801b481b005 P 18694435286247378305d101ea5ddee8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:30.787119 11341 ts_tablet_manager.cc:1428] T 0464f198418b485cb9eb6801b481b005 P d212a963434f4673add46800c0a1ea02: Time spent starting tablet: real 0.011s user 0.012s sys 0.001s
I20260926 15:02:30.787129 11339 raft_consensus.cc:729] T 0464f198418b485cb9eb6801b481b005 P 18694435286247378305d101ea5ddee8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18694435286247378305d101ea5ddee8, State: Initialized, Role: FOLLOWER
I20260926 15:02:30.786984 11338 raft_consensus.cc:348] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:30.787391 11338 raft_consensus.cc:374] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:30.787490 11338 raft_consensus.cc:729] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f66bf759ff54616807d15a79d4d7a88, State: Initialized, Role: FOLLOWER
I20260926 15:02:30.787840 11339 consensus_queue.cc:260] T 0464f198418b485cb9eb6801b481b005 P 18694435286247378305d101ea5ddee8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:30.788023 11338 consensus_queue.cc:260] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:30.789665 11339 ts_tablet_manager.cc:1428] T 0464f198418b485cb9eb6801b481b005 P 18694435286247378305d101ea5ddee8: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260926 15:02:30.791543 11180 heartbeater.cc:503] Master 127.9.123.126:41757 was elected leader, sending a full tablet report...
I20260926 15:02:30.791576 11338 ts_tablet_manager.cc:1428] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88: Time spent starting tablet: real 0.015s user 0.009s sys 0.008s
W20260926 15:02:30.879629 11042 tablet.cc:2369] T 0464f198418b485cb9eb6801b481b005 P 18694435286247378305d101ea5ddee8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:02:30.881618 11346 raft_consensus.cc:482] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:30.881897 11346 raft_consensus.cc:504] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:30.884069 11346 leader_election.cc:288] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18694435286247378305d101ea5ddee8 (127.9.123.65:46371), d212a963434f4673add46800c0a1ea02 (127.9.123.67:41459)
W20260926 15:02:30.886116 11181 tablet.cc:2369] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 15:02:30.891497 11320 tablet.cc:2369] T 0464f198418b485cb9eb6801b481b005 P d212a963434f4673add46800c0a1ea02: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:02:30.892220 11336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:30.884437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:36491 (local address 127.9.123.65:46371)
0926 15:02:30.884647 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:30.884654 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:30.885525 (+ 871us) server_negotiation.cc:407] Connection header received
0926 15:02:30.885680 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:30.885684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:30.885753 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:30.885861 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:30.886564 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.887519 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.890052 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.891238 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.891403 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:30.891510 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:30.891850 (+ 340us) server_negotiation.cc:299] Negotiation successful
0926 15:02:30.891986 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 15:02:30.892364 11348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:30.884459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:46371 (local address 127.9.123.66:36491)
0926 15:02:30.885417 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:30.885438 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:30.885570 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:30.885890 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:30.885893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:30.885941 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:30.886405 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:30.886419 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.887654 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.887659 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:30.889880 (+ 2221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:30.889897 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.891484 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.891492 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:30.891668 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:30.891711 (+ 43us) client_negotiation.cc:769] Sending connection context
0926 15:02:30.891823 (+ 112us) client_negotiation.cc:240] Negotiation successful
0926 15:02:30.891989 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"mutex_wait_us":78,"thread_start_us":637,"threads_started":1}
I20260926 15:02:30.892393 11335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:30.884768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:38961 (local address 127.9.123.67:41459)
0926 15:02:30.884947 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:30.884952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:30.885314 (+ 362us) server_negotiation.cc:407] Connection header received
0926 15:02:30.885513 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:30.885519 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:30.885589 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:30.885671 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:30.886272 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.887270 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.890049 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.891244 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.891419 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:30.891499 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:30.892050 (+ 551us) server_negotiation.cc:299] Negotiation successful
0926 15:02:30.892180 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 15:02:30.893013 11347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:30.884362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:41459 (local address 127.9.123.66:38961)
0926 15:02:30.885188 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:30.885210 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:30.885370 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:30.885696 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:30.885700 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:30.885722 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:30.886124 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:30.886137 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.887428 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.887432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:30.889881 (+ 2449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:30.889897 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.891659 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.891666 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:30.891852 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:30.891885 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 15:02:30.892552 (+ 667us) client_negotiation.cc:240] Negotiation successful
0926 15:02:30.892688 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"mutex_wait_us":91,"thread_start_us":439,"threads_started":1}
I20260926 15:02:30.893978 10996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0464f198418b485cb9eb6801b481b005" candidate_uuid: "4f66bf759ff54616807d15a79d4d7a88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18694435286247378305d101ea5ddee8" is_pre_election: true
I20260926 15:02:30.894090 11273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0464f198418b485cb9eb6801b481b005" candidate_uuid: "4f66bf759ff54616807d15a79d4d7a88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d212a963434f4673add46800c0a1ea02" is_pre_election: true
I20260926 15:02:30.894539 10996 raft_consensus.cc:2454] T 0464f198418b485cb9eb6801b481b005 P 18694435286247378305d101ea5ddee8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f66bf759ff54616807d15a79d4d7a88 in term 0.
I20260926 15:02:30.894606 11273 raft_consensus.cc:2454] T 0464f198418b485cb9eb6801b481b005 P d212a963434f4673add46800c0a1ea02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f66bf759ff54616807d15a79d4d7a88 in term 0.
I20260926 15:02:30.895177 11063 leader_election.cc:302] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18694435286247378305d101ea5ddee8, 4f66bf759ff54616807d15a79d4d7a88; no voters:
I20260926 15:02:30.895548 11346 raft_consensus.cc:2781] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:30.895694 11346 raft_consensus.cc:482] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:30.895797 11346 raft_consensus.cc:3037] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:30.898087 11346 raft_consensus.cc:504] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:30.898717 11346 leader_election.cc:288] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88 [CANDIDATE]: Term 1 election: Requested vote from peers 18694435286247378305d101ea5ddee8 (127.9.123.65:46371), d212a963434f4673add46800c0a1ea02 (127.9.123.67:41459)
I20260926 15:02:30.899245 11273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0464f198418b485cb9eb6801b481b005" candidate_uuid: "4f66bf759ff54616807d15a79d4d7a88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d212a963434f4673add46800c0a1ea02"
I20260926 15:02:30.899256 10996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0464f198418b485cb9eb6801b481b005" candidate_uuid: "4f66bf759ff54616807d15a79d4d7a88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18694435286247378305d101ea5ddee8"
I20260926 15:02:30.899431 11273 raft_consensus.cc:3037] T 0464f198418b485cb9eb6801b481b005 P d212a963434f4673add46800c0a1ea02 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:30.899457 10996 raft_consensus.cc:3037] T 0464f198418b485cb9eb6801b481b005 P 18694435286247378305d101ea5ddee8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:30.901726 11273 raft_consensus.cc:2454] T 0464f198418b485cb9eb6801b481b005 P d212a963434f4673add46800c0a1ea02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f66bf759ff54616807d15a79d4d7a88 in term 1.
I20260926 15:02:30.901752 10996 raft_consensus.cc:2454] T 0464f198418b485cb9eb6801b481b005 P 18694435286247378305d101ea5ddee8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f66bf759ff54616807d15a79d4d7a88 in term 1.
I20260926 15:02:30.902213 11061 leader_election.cc:302] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f66bf759ff54616807d15a79d4d7a88, d212a963434f4673add46800c0a1ea02; no voters:
I20260926 15:02:30.902526 11346 raft_consensus.cc:2781] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:30.903002 11346 raft_consensus.cc:686] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88 [term 1 LEADER]: Becoming Leader. State: Replica: 4f66bf759ff54616807d15a79d4d7a88, State: Running, Role: LEADER
I20260926 15:02:30.903471 11346 consensus_queue.cc:237] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:30.908294 10834 catalog_manager.cc:5495] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f66bf759ff54616807d15a79d4d7a88 (127.9.123.66). New cstate: current_term: 1 leader_uuid: "4f66bf759ff54616807d15a79d4d7a88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } health_report { overall_health: UNKNOWN } } }
I20260926 15:02:30.949415 11333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:30.943992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48916 (local address 127.9.123.66:33611)
0926 15:02:30.944192 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:30.944198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:30.944301 (+ 103us) server_negotiation.cc:407] Connection header received
0926 15:02:30.944614 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:30.944618 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:30.944693 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:30.944814 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:02:30.945185 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.945900 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.947224 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:30.947638 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:30.947745 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:30.948242 (+ 497us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:02:30.948782 (+ 540us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:02:30.948911 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:30.949045 (+ 134us) server_negotiation.cc:299] Negotiation successful
0926 15:02:30.949155 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 15:02:30.958397 11273 raft_consensus.cc:1261] T 0464f198418b485cb9eb6801b481b005 P d212a963434f4673add46800c0a1ea02 [term 1 FOLLOWER]: Refusing update from remote peer 4f66bf759ff54616807d15a79d4d7a88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:02:30.958398 10996 raft_consensus.cc:1261] T 0464f198418b485cb9eb6801b481b005 P 18694435286247378305d101ea5ddee8 [term 1 FOLLOWER]: Refusing update from remote peer 4f66bf759ff54616807d15a79d4d7a88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:02:30.959461 11349 consensus_queue.cc:1035] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:30.959812 11346 consensus_queue.cc:1035] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:30.978363 11353 mvcc.cc:205] Tried to move back new op lower bound from 7333621559114711040 to 7333621558904500224. Current Snapshot: MvccSnapshot[applied={T|T < 7333621559114711040}]
I20260926 15:02:30.993572 10833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:02:30.993928 10833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:02:30.997982 10833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1: Sending DeleteTablet for 3 replicas of tablet 0464f198418b485cb9eb6801b481b005
I20260926 15:02:30.999295 11249 tablet_service.cc:1514] Processing DeleteTablet for tablet 0464f198418b485cb9eb6801b481b005 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:02:30 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:48174
I20260926 15:02:30.999301 11110 tablet_service.cc:1514] Processing DeleteTablet for tablet 0464f198418b485cb9eb6801b481b005 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:02:30 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:48902
I20260926 15:02:30.999557 10971 tablet_service.cc:1514] Processing DeleteTablet for tablet 0464f198418b485cb9eb6801b481b005 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:02:30 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:57050
I20260926 15:02:31.002049 10833 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:02:31.002805 10833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:02:31.014237 11248 tablet_service.cc:1467] Processing CreateTablet for tablet 79d9601a17604423b6feb9b6869c8c95 (DEFAULT_TABLE table=test-table [id=f34222fc8c184d85be7a373b52f9dd02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:31.014231 10970 tablet_service.cc:1467] Processing CreateTablet for tablet 79d9601a17604423b6feb9b6869c8c95 (DEFAULT_TABLE table=test-table [id=f34222fc8c184d85be7a373b52f9dd02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:31.014792 11248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79d9601a17604423b6feb9b6869c8c95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:31.014792 10970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79d9601a17604423b6feb9b6869c8c95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:31.018244 11109 tablet_service.cc:1467] Processing CreateTablet for tablet 79d9601a17604423b6feb9b6869c8c95 (DEFAULT_TABLE table=test-table [id=f34222fc8c184d85be7a373b52f9dd02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:31.021019 11109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79d9601a17604423b6feb9b6869c8c95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:31.024856 11341 tablet_bootstrap.cc:492] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02: Bootstrap starting.
I20260926 15:02:31.026636 11341 tablet_bootstrap.cc:654] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:31.038182 11339 tablet_bootstrap.cc:492] T 79d9601a17604423b6feb9b6869c8c95 P 18694435286247378305d101ea5ddee8: Bootstrap starting.
I20260926 15:02:31.039084 11338 tablet_bootstrap.cc:492] T 79d9601a17604423b6feb9b6869c8c95 P 4f66bf759ff54616807d15a79d4d7a88: Bootstrap starting.
I20260926 15:02:31.039917 11339 tablet_bootstrap.cc:654] T 79d9601a17604423b6feb9b6869c8c95 P 18694435286247378305d101ea5ddee8: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:31.040164 11341 tablet_bootstrap.cc:492] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02: No bootstrap required, opened a new log
I20260926 15:02:31.040385 11341 ts_tablet_manager.cc:1397] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.006s
I20260926 15:02:31.040657 11338 tablet_bootstrap.cc:654] T 79d9601a17604423b6feb9b6869c8c95 P 4f66bf759ff54616807d15a79d4d7a88: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:31.041743 11341 raft_consensus.cc:348] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:31.042145 11341 raft_consensus.cc:374] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:31.042290 11341 raft_consensus.cc:729] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d212a963434f4673add46800c0a1ea02, State: Initialized, Role: FOLLOWER
I20260926 15:02:31.042721 11341 consensus_queue.cc:260] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:31.044683 11364 tablet_replica.cc:330] T 0464f198418b485cb9eb6801b481b005 P 18694435286247378305d101ea5ddee8: stopping tablet replica
I20260926 15:02:31.045357 11364 raft_consensus.cc:2229] T 0464f198418b485cb9eb6801b481b005 P 18694435286247378305d101ea5ddee8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:31.045884 11364 raft_consensus.cc:2258] T 0464f198418b485cb9eb6801b481b005 P 18694435286247378305d101ea5ddee8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:31.046166 11341 ts_tablet_manager.cc:1428] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260926 15:02:31.049973 11364 ts_tablet_manager.cc:1905] T 0464f198418b485cb9eb6801b481b005 P 18694435286247378305d101ea5ddee8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:31.050462 11362 tablet_replica.cc:330] T 0464f198418b485cb9eb6801b481b005 P d212a963434f4673add46800c0a1ea02: stopping tablet replica
I20260926 15:02:31.051014 11362 raft_consensus.cc:2229] T 0464f198418b485cb9eb6801b481b005 P d212a963434f4673add46800c0a1ea02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:31.051647 11362 raft_consensus.cc:2258] T 0464f198418b485cb9eb6801b481b005 P d212a963434f4673add46800c0a1ea02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:31.054265 11339 tablet_bootstrap.cc:492] T 79d9601a17604423b6feb9b6869c8c95 P 18694435286247378305d101ea5ddee8: No bootstrap required, opened a new log
I20260926 15:02:31.054430 11339 ts_tablet_manager.cc:1397] T 79d9601a17604423b6feb9b6869c8c95 P 18694435286247378305d101ea5ddee8: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.004s
I20260926 15:02:31.054832 11362 ts_tablet_manager.cc:1905] T 0464f198418b485cb9eb6801b481b005 P d212a963434f4673add46800c0a1ea02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:31.055246 11339 raft_consensus.cc:348] T 79d9601a17604423b6feb9b6869c8c95 P 18694435286247378305d101ea5ddee8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:31.055457 11339 raft_consensus.cc:374] T 79d9601a17604423b6feb9b6869c8c95 P 18694435286247378305d101ea5ddee8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:31.055540 11339 raft_consensus.cc:729] T 79d9601a17604423b6feb9b6869c8c95 P 18694435286247378305d101ea5ddee8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18694435286247378305d101ea5ddee8, State: Initialized, Role: FOLLOWER
I20260926 15:02:31.055791 11339 consensus_queue.cc:260] T 79d9601a17604423b6feb9b6869c8c95 P 18694435286247378305d101ea5ddee8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:31.058627 11363 tablet_replica.cc:330] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88: stopping tablet replica
I20260926 15:02:31.059096 11363 raft_consensus.cc:2229] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88 [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:02:31.059561 11339 ts_tablet_manager.cc:1428] T 79d9601a17604423b6feb9b6869c8c95 P 18694435286247378305d101ea5ddee8: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260926 15:02:31.059842 11363 raft_consensus.cc:2258] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:31.061671 11362 ts_tablet_manager.cc:1918] T 0464f198418b485cb9eb6801b481b005 P d212a963434f4673add46800c0a1ea02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:02:31.061848 11362 log.cc:1196] T 0464f198418b485cb9eb6801b481b005 P d212a963434f4673add46800c0a1ea02: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/0464f198418b485cb9eb6801b481b005
I20260926 15:02:31.062327 11362 ts_tablet_manager.cc:1939] T 0464f198418b485cb9eb6801b481b005 P d212a963434f4673add46800c0a1ea02: Deleting consensus metadata
I20260926 15:02:31.062929 11364 ts_tablet_manager.cc:1918] T 0464f198418b485cb9eb6801b481b005 P 18694435286247378305d101ea5ddee8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:02:31.064576 10818 catalog_manager.cc:4841] TS d212a963434f4673add46800c0a1ea02 (127.9.123.67:41459): tablet 0464f198418b485cb9eb6801b481b005 (table test-table [id=510f254776a5474489121f5fceb35d57]) successfully deleted
I20260926 15:02:31.065589 11363 ts_tablet_manager.cc:1905] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:31.065632 11338 tablet_bootstrap.cc:492] T 79d9601a17604423b6feb9b6869c8c95 P 4f66bf759ff54616807d15a79d4d7a88: No bootstrap required, opened a new log
I20260926 15:02:31.065815 11338 ts_tablet_manager.cc:1397] T 79d9601a17604423b6feb9b6869c8c95 P 4f66bf759ff54616807d15a79d4d7a88: Time spent bootstrapping tablet: real 0.027s user 0.008s sys 0.003s
I20260926 15:02:31.066754 11338 raft_consensus.cc:348] T 79d9601a17604423b6feb9b6869c8c95 P 4f66bf759ff54616807d15a79d4d7a88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:31.067021 11338 raft_consensus.cc:374] T 79d9601a17604423b6feb9b6869c8c95 P 4f66bf759ff54616807d15a79d4d7a88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:31.067131 11338 raft_consensus.cc:729] T 79d9601a17604423b6feb9b6869c8c95 P 4f66bf759ff54616807d15a79d4d7a88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f66bf759ff54616807d15a79d4d7a88, State: Initialized, Role: FOLLOWER
I20260926 15:02:31.067416 11338 consensus_queue.cc:260] T 79d9601a17604423b6feb9b6869c8c95 P 4f66bf759ff54616807d15a79d4d7a88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:31.068087 11338 ts_tablet_manager.cc:1428] T 79d9601a17604423b6feb9b6869c8c95 P 4f66bf759ff54616807d15a79d4d7a88: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260926 15:02:31.071977 11364 log.cc:1196] T 0464f198418b485cb9eb6801b481b005 P 18694435286247378305d101ea5ddee8: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-0/wal/wals/0464f198418b485cb9eb6801b481b005
I20260926 15:02:31.072659 11364 ts_tablet_manager.cc:1939] T 0464f198418b485cb9eb6801b481b005 P 18694435286247378305d101ea5ddee8: Deleting consensus metadata
I20260926 15:02:31.074395 10820 catalog_manager.cc:4841] TS 18694435286247378305d101ea5ddee8 (127.9.123.65:46371): tablet 0464f198418b485cb9eb6801b481b005 (table test-table [id=510f254776a5474489121f5fceb35d57]) successfully deleted
I20260926 15:02:31.075692 11363 ts_tablet_manager.cc:1918] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:02:31.075889 11363 log.cc:1196] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/0464f198418b485cb9eb6801b481b005
I20260926 15:02:31.076308 11363 ts_tablet_manager.cc:1939] T 0464f198418b485cb9eb6801b481b005 P 4f66bf759ff54616807d15a79d4d7a88: Deleting consensus metadata
I20260926 15:02:31.078377 10820 catalog_manager.cc:4841] TS 4f66bf759ff54616807d15a79d4d7a88 (127.9.123.66:33611): tablet 0464f198418b485cb9eb6801b481b005 (table test-table [id=510f254776a5474489121f5fceb35d57]) successfully deleted
I20260926 15:02:31.222777 11344 raft_consensus.cc:482] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:31.223008 11344 raft_consensus.cc:504] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:31.224704 11344 leader_election.cc:288] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18694435286247378305d101ea5ddee8 (127.9.123.65:46371), 4f66bf759ff54616807d15a79d4d7a88 (127.9.123.66:33611)
I20260926 15:02:31.231464 11336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:31.225172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:39545 (local address 127.9.123.65:46371)
0926 15:02:31.225354 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:31.225360 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:31.225575 (+ 215us) server_negotiation.cc:407] Connection header received
0926 15:02:31.225827 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:31.225833 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:31.225901 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:31.225958 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:31.226635 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.227640 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.229353 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.230542 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.230709 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:31.230782 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:31.231052 (+ 270us) server_negotiation.cc:299] Negotiation successful
0926 15:02:31.231182 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 15:02:31.231775 11368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:31.224919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:46371 (local address 127.9.123.67:39545)
0926 15:02:31.225462 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:31.225483 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:31.225684 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:31.226153 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:31.226157 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:31.226181 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:31.226464 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:31.226474 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.227750 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.227755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:31.229186 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:31.229200 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.230707 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.230714 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:31.230888 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:31.230923 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 15:02:31.231015 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 15:02:31.231466 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":297,"threads_started":1}
I20260926 15:02:31.232445 10996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79d9601a17604423b6feb9b6869c8c95" candidate_uuid: "d212a963434f4673add46800c0a1ea02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18694435286247378305d101ea5ddee8" is_pre_election: true
I20260926 15:02:31.232707 10996 raft_consensus.cc:2454] T 79d9601a17604423b6feb9b6869c8c95 P 18694435286247378305d101ea5ddee8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d212a963434f4673add46800c0a1ea02 in term 0.
I20260926 15:02:31.233520 11333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:31.225873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:35219 (local address 127.9.123.66:33611)
0926 15:02:31.226081 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:31.226087 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:31.226593 (+ 506us) server_negotiation.cc:407] Connection header received
0926 15:02:31.226842 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:31.226847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:31.226922 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:31.227007 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:31.227504 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.228572 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.230304 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.231418 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.231565 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:31.231636 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:31.232555 (+ 919us) server_negotiation.cc:299] Negotiation successful
0926 15:02:31.232708 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 15:02:31.233522 11369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:31.225614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:33611 (local address 127.9.123.67:35219)
0926 15:02:31.226500 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:31.226514 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:31.226681 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:31.227042 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:31.227047 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:31.227073 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:31.227337 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:31.227347 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.228725 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.228731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:31.230160 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:31.230170 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.232188 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.232193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:31.232350 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:31.232380 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 15:02:31.233062 (+ 682us) client_negotiation.cc:240] Negotiation successful
0926 15:02:31.233214 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":800,"thread_start_us":630,"threads_started":1}
I20260926 15:02:31.233438 11202 leader_election.cc:302] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18694435286247378305d101ea5ddee8, d212a963434f4673add46800c0a1ea02; no voters:
I20260926 15:02:31.233994 11344 raft_consensus.cc:2781] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:31.234112 11344 raft_consensus.cc:482] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:31.234200 11344 raft_consensus.cc:3037] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:31.234555 11130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79d9601a17604423b6feb9b6869c8c95" candidate_uuid: "d212a963434f4673add46800c0a1ea02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f66bf759ff54616807d15a79d4d7a88" is_pre_election: true
I20260926 15:02:31.234911 11130 raft_consensus.cc:2454] T 79d9601a17604423b6feb9b6869c8c95 P 4f66bf759ff54616807d15a79d4d7a88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d212a963434f4673add46800c0a1ea02 in term 0.
I20260926 15:02:31.237002 11344 raft_consensus.cc:504] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:31.237699 11344 leader_election.cc:288] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02 [CANDIDATE]: Term 1 election: Requested vote from peers 18694435286247378305d101ea5ddee8 (127.9.123.65:46371), 4f66bf759ff54616807d15a79d4d7a88 (127.9.123.66:33611)
I20260926 15:02:31.238204 11130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79d9601a17604423b6feb9b6869c8c95" candidate_uuid: "d212a963434f4673add46800c0a1ea02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f66bf759ff54616807d15a79d4d7a88"
I20260926 15:02:31.238204 10996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79d9601a17604423b6feb9b6869c8c95" candidate_uuid: "d212a963434f4673add46800c0a1ea02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18694435286247378305d101ea5ddee8"
I20260926 15:02:31.238385 11130 raft_consensus.cc:3037] T 79d9601a17604423b6feb9b6869c8c95 P 4f66bf759ff54616807d15a79d4d7a88 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:31.238412 10996 raft_consensus.cc:3037] T 79d9601a17604423b6feb9b6869c8c95 P 18694435286247378305d101ea5ddee8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:31.240598 11130 raft_consensus.cc:2454] T 79d9601a17604423b6feb9b6869c8c95 P 4f66bf759ff54616807d15a79d4d7a88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d212a963434f4673add46800c0a1ea02 in term 1.
I20260926 15:02:31.240984 11202 leader_election.cc:302] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f66bf759ff54616807d15a79d4d7a88, d212a963434f4673add46800c0a1ea02; no voters:
I20260926 15:02:31.241289 11344 raft_consensus.cc:2781] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:31.241331 10996 raft_consensus.cc:2454] T 79d9601a17604423b6feb9b6869c8c95 P 18694435286247378305d101ea5ddee8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d212a963434f4673add46800c0a1ea02 in term 1.
I20260926 15:02:31.241818 11344 raft_consensus.cc:686] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02 [term 1 LEADER]: Becoming Leader. State: Replica: d212a963434f4673add46800c0a1ea02, State: Running, Role: LEADER
I20260926 15:02:31.242192 11344 consensus_queue.cc:237] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:31.245812 10834 catalog_manager.cc:5495] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02 reported cstate change: term changed from 0 to 1, leader changed from <none> to d212a963434f4673add46800c0a1ea02 (127.9.123.67). New cstate: current_term: 1 leader_uuid: "d212a963434f4673add46800c0a1ea02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } health_report { overall_health: HEALTHY } } }
Sep 26 15:02:31 dist-test-slave-7153 krb5kdc[10789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434948, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:31.330581 11378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:31.315517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60720 (local address 127.9.123.126:41757)
0926 15:02:31.316412 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:31.316417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:31.316436 (+ 19us) server_negotiation.cc:407] Connection header received
0926 15:02:31.316475 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:31.316478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:31.316530 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:31.316613 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:31.317739 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.318493 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.320114 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.320514 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.320619 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:31.324808 (+ 4189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:31.324813 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:31.324816 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:31.324847 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:31.327559 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:31.328547 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:31.328551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:31.328553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:31.328672 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:31.329056 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:31.329059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:31.329061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:31.329350 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:31.329473 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:31.329952 (+ 479us) server_negotiation.cc:299] Negotiation successful
0926 15:02:31.330282 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":156,"server-negotiator.queue_time_us":775,"thread_start_us":628,"threads_started":1}
I20260926 15:02:31.330621 11377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:31.315488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:41757 (local address 127.0.0.1:60720)
0926 15:02:31.316105 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:31.316130 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:31.316288 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:31.316661 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:31.316665 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:31.317396 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:31.317589 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:31.317599 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.318614 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.318618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:31.319977 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:31.319989 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.320668 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.320674 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:31.320833 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:31.321621 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:31.321640 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:31.324327 (+ 2687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:31.327722 (+ 3395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:31.327743 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:31.327754 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:31.328419 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:31.328777 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:31.328781 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:31.328785 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:31.328959 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:31.329489 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:31.329498 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:31.329694 (+ 196us) client_negotiation.cc:769] Sending connection context
0926 15:02:31.329885 (+ 191us) client_negotiation.cc:240] Negotiation successful
0926 15:02:31.330189 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":305,"threads_started":1}
I20260926 15:02:31.335634 10834 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:60720:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 15:02:31.336639 10834 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:02:31.343915 10971 tablet_service.cc:1467] Processing CreateTablet for tablet 81cb301a85b44a949f70b9a16b7ee60d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3f6ad3657b4b4bcd85992dd85df2432c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 15:02:31.344044 11110 tablet_service.cc:1467] Processing CreateTablet for tablet 81cb301a85b44a949f70b9a16b7ee60d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3f6ad3657b4b4bcd85992dd85df2432c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 15:02:31.344147 11249 tablet_service.cc:1467] Processing CreateTablet for tablet 81cb301a85b44a949f70b9a16b7ee60d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3f6ad3657b4b4bcd85992dd85df2432c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 15:02:31.344542 10971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81cb301a85b44a949f70b9a16b7ee60d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:31.344609 11110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81cb301a85b44a949f70b9a16b7ee60d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:31.344620 11249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81cb301a85b44a949f70b9a16b7ee60d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:31.351512 11339 tablet_bootstrap.cc:492] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8: Bootstrap starting.
I20260926 15:02:31.351512 11341 tablet_bootstrap.cc:492] T 81cb301a85b44a949f70b9a16b7ee60d P d212a963434f4673add46800c0a1ea02: Bootstrap starting.
I20260926 15:02:31.353170 11339 tablet_bootstrap.cc:654] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:31.353250 11341 tablet_bootstrap.cc:654] T 81cb301a85b44a949f70b9a16b7ee60d P d212a963434f4673add46800c0a1ea02: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:31.354694 11338 tablet_bootstrap.cc:492] T 81cb301a85b44a949f70b9a16b7ee60d P 4f66bf759ff54616807d15a79d4d7a88: Bootstrap starting.
I20260926 15:02:31.356571 11338 tablet_bootstrap.cc:654] T 81cb301a85b44a949f70b9a16b7ee60d P 4f66bf759ff54616807d15a79d4d7a88: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:31.358997 11341 tablet_bootstrap.cc:492] T 81cb301a85b44a949f70b9a16b7ee60d P d212a963434f4673add46800c0a1ea02: No bootstrap required, opened a new log
I20260926 15:02:31.359170 11341 ts_tablet_manager.cc:1397] T 81cb301a85b44a949f70b9a16b7ee60d P d212a963434f4673add46800c0a1ea02: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260926 15:02:31.359555 11339 tablet_bootstrap.cc:492] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8: No bootstrap required, opened a new log
I20260926 15:02:31.359707 11339 ts_tablet_manager.cc:1397] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260926 15:02:31.359925 11341 raft_consensus.cc:348] T 81cb301a85b44a949f70b9a16b7ee60d P d212a963434f4673add46800c0a1ea02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:31.360234 11341 raft_consensus.cc:374] T 81cb301a85b44a949f70b9a16b7ee60d P d212a963434f4673add46800c0a1ea02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:31.360327 11341 raft_consensus.cc:729] T 81cb301a85b44a949f70b9a16b7ee60d P d212a963434f4673add46800c0a1ea02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d212a963434f4673add46800c0a1ea02, State: Initialized, Role: FOLLOWER
I20260926 15:02:31.360560 11341 consensus_queue.cc:260] T 81cb301a85b44a949f70b9a16b7ee60d P d212a963434f4673add46800c0a1ea02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:31.360980 11344 tablet_replica.cc:439] T 81cb301a85b44a949f70b9a16b7ee60d P d212a963434f4673add46800c0a1ea02: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } } }
I20260926 15:02:31.361292 11344 tablet_replica.cc:442] T 81cb301a85b44a949f70b9a16b7ee60d P d212a963434f4673add46800c0a1ea02: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 15:02:31.361740 11341 ts_tablet_manager.cc:1428] T 81cb301a85b44a949f70b9a16b7ee60d P d212a963434f4673add46800c0a1ea02: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 15:02:31.362728 11339 raft_consensus.cc:348] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:31.362941 11339 raft_consensus.cc:374] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:31.362990 11339 raft_consensus.cc:729] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18694435286247378305d101ea5ddee8, State: Initialized, Role: FOLLOWER
I20260926 15:02:31.363184 11339 consensus_queue.cc:260] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:31.363564 11345 tablet_replica.cc:439] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } } }
I20260926 15:02:31.363878 11345 tablet_replica.cc:442] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 15:02:31.364696 11339 ts_tablet_manager.cc:1428] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260926 15:02:31.366506 11338 tablet_bootstrap.cc:492] T 81cb301a85b44a949f70b9a16b7ee60d P 4f66bf759ff54616807d15a79d4d7a88: No bootstrap required, opened a new log
I20260926 15:02:31.366668 11338 ts_tablet_manager.cc:1397] T 81cb301a85b44a949f70b9a16b7ee60d P 4f66bf759ff54616807d15a79d4d7a88: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.001s
I20260926 15:02:31.367496 11338 raft_consensus.cc:348] T 81cb301a85b44a949f70b9a16b7ee60d P 4f66bf759ff54616807d15a79d4d7a88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:31.367730 11338 raft_consensus.cc:374] T 81cb301a85b44a949f70b9a16b7ee60d P 4f66bf759ff54616807d15a79d4d7a88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:31.367812 11338 raft_consensus.cc:729] T 81cb301a85b44a949f70b9a16b7ee60d P 4f66bf759ff54616807d15a79d4d7a88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f66bf759ff54616807d15a79d4d7a88, State: Initialized, Role: FOLLOWER
I20260926 15:02:31.368055 11338 consensus_queue.cc:260] T 81cb301a85b44a949f70b9a16b7ee60d P 4f66bf759ff54616807d15a79d4d7a88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:31.368786 11357 tablet_replica.cc:439] T 81cb301a85b44a949f70b9a16b7ee60d P 4f66bf759ff54616807d15a79d4d7a88: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } } }
I20260926 15:02:31.369186 11338 ts_tablet_manager.cc:1428] T 81cb301a85b44a949f70b9a16b7ee60d P 4f66bf759ff54616807d15a79d4d7a88: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 15:02:31.369239 11357 tablet_replica.cc:442] T 81cb301a85b44a949f70b9a16b7ee60d P 4f66bf759ff54616807d15a79d4d7a88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 15:02:31.576020 11345 raft_consensus.cc:482] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:31.576212 11345 raft_consensus.cc:504] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:31.577657 11345 leader_election.cc:288] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f66bf759ff54616807d15a79d4d7a88 (127.9.123.66:33611), d212a963434f4673add46800c0a1ea02 (127.9.123.67:41459)
I20260926 15:02:31.583763 11333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:31.577953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:42331 (local address 127.9.123.66:33611)
0926 15:02:31.578221 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:31.578229 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:31.578678 (+ 449us) server_negotiation.cc:407] Connection header received
0926 15:02:31.578857 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:31.578860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:31.578920 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:31.578992 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:31.579429 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.580081 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.581715 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.582741 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.582906 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:31.582981 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:31.583289 (+ 308us) server_negotiation.cc:299] Negotiation successful
0926 15:02:31.583433 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260926 15:02:31.584489 11384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:31.578187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:43365 (local address 127.9.123.67:41459)
0926 15:02:31.579175 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:31.579180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:31.579194 (+ 14us) server_negotiation.cc:407] Connection header received
0926 15:02:31.579243 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:31.579246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:31.579295 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:31.579376 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:31.579870 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.580572 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.582200 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.583274 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.583427 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:31.583509 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:31.584096 (+ 587us) server_negotiation.cc:299] Negotiation successful
0926 15:02:31.584235 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":900,"thread_start_us":806,"threads_started":1}
I20260926 15:02:31.584474 11382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:31.577890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:33611 (local address 127.9.123.65:42331)
0926 15:02:31.578570 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:31.578587 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:31.578726 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:31.579020 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:31.579025 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:31.579049 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:31.579289 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:31.579297 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.580206 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.580212 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:31.581569 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:31.581582 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.582908 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.582913 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:31.583088 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:31.583120 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 15:02:31.584005 (+ 885us) client_negotiation.cc:240] Negotiation successful
0926 15:02:31.584129 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":471,"threads_started":1}
I20260926 15:02:31.584494 11383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:31.577895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:41459 (local address 127.9.123.65:43365)
0926 15:02:31.578692 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:31.578711 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:31.578847 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:31.579409 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:31.579414 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:31.579436 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:31.579735 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:31.579744 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.580689 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.580693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:31.582029 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:31.582041 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.583493 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.583500 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:31.583726 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:31.583762 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 15:02:31.583851 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 15:02:31.583994 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"mutex_wait_us":7,"thread_start_us":454,"threads_started":1}
I20260926 15:02:31.585115 11272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81cb301a85b44a949f70b9a16b7ee60d" candidate_uuid: "18694435286247378305d101ea5ddee8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d212a963434f4673add46800c0a1ea02" is_pre_election: true
I20260926 15:02:31.585376 11272 raft_consensus.cc:2454] T 81cb301a85b44a949f70b9a16b7ee60d P d212a963434f4673add46800c0a1ea02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18694435286247378305d101ea5ddee8 in term 0.
I20260926 15:02:31.585371 11130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81cb301a85b44a949f70b9a16b7ee60d" candidate_uuid: "18694435286247378305d101ea5ddee8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f66bf759ff54616807d15a79d4d7a88" is_pre_election: true
I20260926 15:02:31.585595 11130 raft_consensus.cc:2454] T 81cb301a85b44a949f70b9a16b7ee60d P 4f66bf759ff54616807d15a79d4d7a88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18694435286247378305d101ea5ddee8 in term 0.
I20260926 15:02:31.585834 10921 leader_election.cc:302] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18694435286247378305d101ea5ddee8, d212a963434f4673add46800c0a1ea02; no voters:
I20260926 15:02:31.586211 11345 raft_consensus.cc:2781] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:31.586357 11345 raft_consensus.cc:482] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:31.586450 11345 raft_consensus.cc:3037] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:31.588886 11345 raft_consensus.cc:504] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:31.589526 11345 leader_election.cc:288] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8 [CANDIDATE]: Term 1 election: Requested vote from peers 4f66bf759ff54616807d15a79d4d7a88 (127.9.123.66:33611), d212a963434f4673add46800c0a1ea02 (127.9.123.67:41459)
I20260926 15:02:31.589984 11130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81cb301a85b44a949f70b9a16b7ee60d" candidate_uuid: "18694435286247378305d101ea5ddee8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f66bf759ff54616807d15a79d4d7a88"
I20260926 15:02:31.590015 11272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81cb301a85b44a949f70b9a16b7ee60d" candidate_uuid: "18694435286247378305d101ea5ddee8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d212a963434f4673add46800c0a1ea02"
I20260926 15:02:31.590195 11272 raft_consensus.cc:3037] T 81cb301a85b44a949f70b9a16b7ee60d P d212a963434f4673add46800c0a1ea02 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:31.590216 11130 raft_consensus.cc:3037] T 81cb301a85b44a949f70b9a16b7ee60d P 4f66bf759ff54616807d15a79d4d7a88 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:31.592967 11130 raft_consensus.cc:2454] T 81cb301a85b44a949f70b9a16b7ee60d P 4f66bf759ff54616807d15a79d4d7a88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18694435286247378305d101ea5ddee8 in term 1.
I20260926 15:02:31.592991 11272 raft_consensus.cc:2454] T 81cb301a85b44a949f70b9a16b7ee60d P d212a963434f4673add46800c0a1ea02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18694435286247378305d101ea5ddee8 in term 1.
I20260926 15:02:31.593408 10923 leader_election.cc:302] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18694435286247378305d101ea5ddee8, 4f66bf759ff54616807d15a79d4d7a88; no voters:
I20260926 15:02:31.593727 11345 raft_consensus.cc:2781] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:31.594107 11345 raft_consensus.cc:686] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8 [term 1 LEADER]: Becoming Leader. State: Replica: 18694435286247378305d101ea5ddee8, State: Running, Role: LEADER
I20260926 15:02:31.594502 11345 consensus_queue.cc:237] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } }
I20260926 15:02:31.596374 11385 tablet_replica.cc:439] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8: TxnStatusTablet state changed. Reason: New leader 18694435286247378305d101ea5ddee8. Latest consensus state: current_term: 1 leader_uuid: "18694435286247378305d101ea5ddee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } } }
I20260926 15:02:31.596736 11385 tablet_replica.cc:442] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8: This TxnStatusTablet replica's current role is: LEADER
I20260926 15:02:31.597427 11387 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 15:02:31.597641 11387 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 15:02:31.598086 10834 catalog_manager.cc:5495] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18694435286247378305d101ea5ddee8 (127.9.123.65). New cstate: current_term: 1 leader_uuid: "18694435286247378305d101ea5ddee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } health_report { overall_health: UNKNOWN } } }
I20260926 15:02:31.647305 11344 consensus_queue.cc:1035] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:31.675770 11377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:31.664917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:46371 (local address 127.0.0.1:57062)
0926 15:02:31.666020 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:31.666095 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:31.666448 (+ 353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:31.667983 (+ 1535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:31.667990 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:31.668006 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 15:02:31.668404 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:31.668421 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.670474 (+ 2053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.670480 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:31.672128 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:31.672146 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.673401 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.673409 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:31.673645 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:31.673742 (+ 97us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 15:02:31.675068 (+ 1326us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 15:02:31.675098 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 15:02:31.675175 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 15:02:31.675380 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223}
I20260926 15:02:31.676673 11336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:31.665623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57062 (local address 127.9.123.65:46371)
0926 15:02:31.665876 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:31.665885 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:31.666233 (+ 348us) server_negotiation.cc:407] Connection header received
0926 15:02:31.667144 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:31.667176 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:31.667361 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:31.667584 (+ 223us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:02:31.668696 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.670236 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.672330 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.673068 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.673227 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:31.673991 (+ 764us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:02:31.674845 (+ 854us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:02:31.676145 (+ 1300us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:31.676235 (+ 90us) server_negotiation.cc:299] Negotiation successful
0926 15:02:31.676348 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 15:02:31.683760 11344 consensus_queue.cc:1035] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:31.683950 11272 raft_consensus.cc:1261] T 81cb301a85b44a949f70b9a16b7ee60d P d212a963434f4673add46800c0a1ea02 [term 1 FOLLOWER]: Refusing update from remote peer 18694435286247378305d101ea5ddee8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:02:31.684840 11385 consensus_queue.cc:1035] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:31.687390 11130 raft_consensus.cc:1261] T 81cb301a85b44a949f70b9a16b7ee60d P 4f66bf759ff54616807d15a79d4d7a88 [term 1 FOLLOWER]: Refusing update from remote peer 18694435286247378305d101ea5ddee8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:02:31.687959 11385 consensus_queue.cc:1035] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:31.690611 11344 tablet_replica.cc:439] T 81cb301a85b44a949f70b9a16b7ee60d P d212a963434f4673add46800c0a1ea02: TxnStatusTablet state changed. Reason: New leader 18694435286247378305d101ea5ddee8. Latest consensus state: current_term: 1 leader_uuid: "18694435286247378305d101ea5ddee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } } }
I20260926 15:02:31.690939 11344 tablet_replica.cc:442] T 81cb301a85b44a949f70b9a16b7ee60d P d212a963434f4673add46800c0a1ea02: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 15:02:31.693477 11357 tablet_replica.cc:439] T 81cb301a85b44a949f70b9a16b7ee60d P 4f66bf759ff54616807d15a79d4d7a88: TxnStatusTablet state changed. Reason: New leader 18694435286247378305d101ea5ddee8. Latest consensus state: current_term: 1 leader_uuid: "18694435286247378305d101ea5ddee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } } }
I20260926 15:02:31.693764 11357 tablet_replica.cc:442] T 81cb301a85b44a949f70b9a16b7ee60d P 4f66bf759ff54616807d15a79d4d7a88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 15:02:31.696729 11393 mvcc.cc:205] Tried to move back new op lower bound from 7333621562089828352 to 7333621561734979584. Current Snapshot: MvccSnapshot[applied={T|T < 7333621562089828352}]
I20260926 15:02:31.706115 11345 tablet_replica.cc:439] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "18694435286247378305d101ea5ddee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } } }
I20260926 15:02:31.706471 11345 tablet_replica.cc:442] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8: This TxnStatusTablet replica's current role is: LEADER
I20260926 15:02:31.709075 11385 tablet_replica.cc:439] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "18694435286247378305d101ea5ddee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } } }
I20260926 15:02:31.709404 11385 tablet_replica.cc:442] T 81cb301a85b44a949f70b9a16b7ee60d P 18694435286247378305d101ea5ddee8: This TxnStatusTablet replica's current role is: LEADER
I20260926 15:02:31.711776 11357 tablet_replica.cc:439] T 81cb301a85b44a949f70b9a16b7ee60d P 4f66bf759ff54616807d15a79d4d7a88: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "18694435286247378305d101ea5ddee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } } }
I20260926 15:02:31.712044 11357 tablet_replica.cc:442] T 81cb301a85b44a949f70b9a16b7ee60d P 4f66bf759ff54616807d15a79d4d7a88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 15:02:31.712987 11397 tablet_replica.cc:439] T 81cb301a85b44a949f70b9a16b7ee60d P d212a963434f4673add46800c0a1ea02: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "18694435286247378305d101ea5ddee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18694435286247378305d101ea5ddee8" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46371 } } peers { permanent_uuid: "4f66bf759ff54616807d15a79d4d7a88" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33611 } } peers { permanent_uuid: "d212a963434f4673add46800c0a1ea02" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 41459 } } }
I20260926 15:02:31.713406 11397 tablet_replica.cc:442] T 81cb301a85b44a949f70b9a16b7ee60d P d212a963434f4673add46800c0a1ea02: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 15:02:31.723206 11384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:31.715126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48186 (local address 127.9.123.67:41459)
0926 15:02:31.715813 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:31.715821 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:31.717155 (+ 1334us) server_negotiation.cc:407] Connection header received
0926 15:02:31.717353 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:31.717356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:31.717434 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:31.717558 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:02:31.718035 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.718706 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.721165 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.721560 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.721648 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:31.722131 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:02:31.722675 (+ 544us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:02:31.722778 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:31.722889 (+ 111us) server_negotiation.cc:299] Negotiation successful
0926 15:02:31.722960 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543}
I20260926 15:02:31.738314 11407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:31.733144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:46371 (local address 127.9.123.67:39179)
0926 15:02:31.733867 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:31.733890 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:31.734046 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:31.734332 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:31.734337 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:31.734348 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 15:02:31.734505 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:31.734514 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.735475 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.735480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:31.736392 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:31.736403 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.737193 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.737199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:31.737339 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:31.737393 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 15:02:31.737779 (+ 386us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 15:02:31.737790 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 15:02:31.737858 (+ 68us) client_negotiation.cc:240] Negotiation successful
0926 15:02:31.737991 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":476,"threads_started":1}
I20260926 15:02:31.738296 11336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:31.733419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:39179 (local address 127.9.123.65:46371)
0926 15:02:31.733612 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:31.733618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:31.733970 (+ 352us) server_negotiation.cc:407] Connection header received
0926 15:02:31.734170 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:31.734174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:31.734235 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:31.734305 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:02:31.734627 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.735332 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.736529 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.737019 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.737152 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:31.737554 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:02:31.737691 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:02:31.737769 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:31.737885 (+ 116us) server_negotiation.cc:299] Negotiation successful
0926 15:02:31.737975 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 15:02:31.779263 11384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:31.772421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:52357 (local address 127.9.123.67:41459)
0926 15:02:31.772631 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:31.772638 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:31.773276 (+ 638us) server_negotiation.cc:407] Connection header received
0926 15:02:31.773442 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:31.773449 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:31.773557 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:31.773657 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:02:31.774268 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.775445 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.777041 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.777766 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.777900 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:31.778469 (+ 569us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:02:31.778618 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:02:31.778710 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:31.778866 (+ 156us) server_negotiation.cc:299] Negotiation successful
0926 15:02:31.778964 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 15:02:31.779664 11410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:31.772337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:41459 (local address 127.9.123.65:52357)
0926 15:02:31.773172 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:31.773191 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:31.773324 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:31.773695 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:31.773700 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:31.773712 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 15:02:31.774080 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:31.774095 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.775571 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.775576 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:31.776864 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:31.776883 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.777977 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.777985 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:31.778198 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:31.778270 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 15:02:31.778732 (+ 462us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 15:02:31.778746 (+ 14us) client_negotiation.cc:769] Sending connection context
0926 15:02:31.778834 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 15:02:31.779018 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":442,"threads_started":1}
I20260926 15:02:31.821352 10833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:02:31.821628 10833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:02:31.825475 10833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4effaa816ce84ebd9e2b0f7dd6f383f1: Sending DeleteTablet for 3 replicas of tablet 79d9601a17604423b6feb9b6869c8c95
I20260926 15:02:31.826578 11249 tablet_service.cc:1514] Processing DeleteTablet for tablet 79d9601a17604423b6feb9b6869c8c95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:02:31 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:48174
I20260926 15:02:31.826582 10971 tablet_service.cc:1514] Processing DeleteTablet for tablet 79d9601a17604423b6feb9b6869c8c95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:02:31 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:57050
I20260926 15:02:31.827445 11110 tablet_service.cc:1514] Processing DeleteTablet for tablet 79d9601a17604423b6feb9b6869c8c95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:02:31 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:48902
I20260926 15:02:31.852052 11415 tablet_replica.cc:330] T 79d9601a17604423b6feb9b6869c8c95 P 4f66bf759ff54616807d15a79d4d7a88: stopping tablet replica
I20260926 15:02:31.852442 11415 raft_consensus.cc:2229] T 79d9601a17604423b6feb9b6869c8c95 P 4f66bf759ff54616807d15a79d4d7a88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:31.853041 11414 tablet_replica.cc:330] T 79d9601a17604423b6feb9b6869c8c95 P 18694435286247378305d101ea5ddee8: stopping tablet replica
I20260926 15:02:31.853418 11415 raft_consensus.cc:2258] T 79d9601a17604423b6feb9b6869c8c95 P 4f66bf759ff54616807d15a79d4d7a88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:31.853860 11414 raft_consensus.cc:2229] T 79d9601a17604423b6feb9b6869c8c95 P 18694435286247378305d101ea5ddee8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:31.854809 11415 ts_tablet_manager.cc:1905] T 79d9601a17604423b6feb9b6869c8c95 P 4f66bf759ff54616807d15a79d4d7a88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:31.855161 11414 raft_consensus.cc:2258] T 79d9601a17604423b6feb9b6869c8c95 P 18694435286247378305d101ea5ddee8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:31.858450 11413 tablet_replica.cc:330] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02: stopping tablet replica
I20260926 15:02:31.858666 11413 raft_consensus.cc:2229] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02 [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:02:31.859411 11414 ts_tablet_manager.cc:1905] T 79d9601a17604423b6feb9b6869c8c95 P 18694435286247378305d101ea5ddee8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:31.859246 11413 raft_consensus.cc:2258] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:31.860846 11413 ts_tablet_manager.cc:1905] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:31.868528 11413 ts_tablet_manager.cc:1918] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 15:02:31.868698 11413 log.cc:1196] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/79d9601a17604423b6feb9b6869c8c95
I20260926 15:02:31.874024 11414 ts_tablet_manager.cc:1918] T 79d9601a17604423b6feb9b6869c8c95 P 18694435286247378305d101ea5ddee8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 15:02:31.874178 11414 log.cc:1196] T 79d9601a17604423b6feb9b6869c8c95 P 18694435286247378305d101ea5ddee8: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-0/wal/wals/79d9601a17604423b6feb9b6869c8c95
I20260926 15:02:31.874598 11414 ts_tablet_manager.cc:1939] T 79d9601a17604423b6feb9b6869c8c95 P 18694435286247378305d101ea5ddee8: Deleting consensus metadata
I20260926 15:02:31.875887 10820 catalog_manager.cc:4841] TS 18694435286247378305d101ea5ddee8 (127.9.123.65:46371): tablet 79d9601a17604423b6feb9b6869c8c95 (table test-table [id=f34222fc8c184d85be7a373b52f9dd02]) successfully deleted
I20260926 15:02:31.877411 11415 ts_tablet_manager.cc:1918] T 79d9601a17604423b6feb9b6869c8c95 P 4f66bf759ff54616807d15a79d4d7a88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 15:02:31.877575 11415 log.cc:1196] T 79d9601a17604423b6feb9b6869c8c95 P 4f66bf759ff54616807d15a79d4d7a88: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/79d9601a17604423b6feb9b6869c8c95
I20260926 15:02:31.877961 11415 ts_tablet_manager.cc:1939] T 79d9601a17604423b6feb9b6869c8c95 P 4f66bf759ff54616807d15a79d4d7a88: Deleting consensus metadata
I20260926 15:02:31.879123 10820 catalog_manager.cc:4841] TS 4f66bf759ff54616807d15a79d4d7a88 (127.9.123.66:33611): tablet 79d9601a17604423b6feb9b6869c8c95 (table test-table [id=f34222fc8c184d85be7a373b52f9dd02]) successfully deleted
I20260926 15:02:31.879604 11413 ts_tablet_manager.cc:1939] T 79d9601a17604423b6feb9b6869c8c95 P d212a963434f4673add46800c0a1ea02: Deleting consensus metadata
Sep 26 15:02:31 dist-test-slave-7153 krb5kdc[10789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434950, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.123.65@KRBTEST.COM
I20260926 15:02:31.880699 10818 catalog_manager.cc:4841] TS d212a963434f4673add46800c0a1ea02 (127.9.123.67:41459): tablet 79d9601a17604423b6feb9b6869c8c95 (table test-table [id=f34222fc8c184d85be7a373b52f9dd02]) successfully deleted
I20260926 15:02:31.890264 11336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:31.849694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57064 (local address 127.9.123.65:46371)
0926 15:02:31.849909 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:31.849925 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:31.861684 (+ 11759us) server_negotiation.cc:407] Connection header received
0926 15:02:31.863786 (+ 2102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:31.863792 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:31.863900 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:31.863991 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:31.870654 (+ 6663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.871876 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.873835 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.874495 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.874600 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:31.881798 (+ 7198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:31.881816 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:31.881837 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:31.881893 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:31.886146 (+ 4253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:31.887241 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:31.887252 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:31.887257 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:31.887443 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:31.887940 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:31.887947 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:31.887951 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:31.888454 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:31.888799 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:31.889571 (+ 772us) server_negotiation.cc:299] Negotiation successful
0926 15:02:31.889829 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260926 15:02:31.891304 11036 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57064
I20260926 15:02:31.923108 11420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:31.896133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60722 (local address 127.9.123.126:41757)
0926 15:02:31.903661 (+ 7528us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:31.903668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:31.903694 (+ 26us) server_negotiation.cc:407] Connection header received
0926 15:02:31.903909 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:31.903917 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:31.904011 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:31.904159 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:31.906042 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.907991 (+ 1949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.910108 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.910815 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.910962 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:31.914794 (+ 3832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:31.914802 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:31.914808 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:31.914862 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:31.918665 (+ 3803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:31.919698 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:31.919705 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:31.919709 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:31.919890 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:31.920324 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:31.920329 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:31.920332 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:31.920728 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:31.922292 (+ 1564us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:31.922448 (+ 156us) server_negotiation.cc:299] Negotiation successful
0926 15:02:31.922713 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7390,"thread_start_us":7240,"threads_started":1}
W20260926 15:02:31.923851 10833 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60722
I20260926 15:02:31.927421 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 10905
I20260926 15:02:31.966229 11336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:31.929739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57068 (local address 127.9.123.65:46371)
0926 15:02:31.929955 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:31.929964 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:31.944652 (+ 14688us) server_negotiation.cc:407] Connection header received
0926 15:02:31.944862 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:31.944868 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:31.944957 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:31.945085 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:31.946732 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.947850 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.949843 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.950650 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.950818 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:31.956566 (+ 5748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:31.956586 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:31.956593 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:31.956654 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:31.961662 (+ 5008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:31.962678 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:31.962685 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:31.962689 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:31.962883 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:31.963418 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:31.963423 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:31.963426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:31.963819 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:31.964082 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:31.965553 (+ 1471us) server_negotiation.cc:299] Negotiation successful
0926 15:02:31.965815 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260926 15:02:31.967038 11036 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57068
W20260926 15:02:31.967672 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 10905: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 15:02:31.967931 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 10905
I20260926 15:02:31.991442 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 11045
Sep 26 15:02:31 dist-test-slave-7153 krb5kdc[10789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434950, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.123.66@KRBTEST.COM
I20260926 15:02:32.007472 11333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:31.992156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48930 (local address 127.9.123.66:33611)
0926 15:02:31.992315 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:31.992321 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:31.992399 (+ 78us) server_negotiation.cc:407] Connection header received
0926 15:02:31.992586 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:31.992591 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:31.992653 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:31.992745 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:31.993919 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.994686 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.995990 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:31.996368 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:31.996474 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:32.000705 (+ 4231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:32.000726 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:32.000748 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:32.000798 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:32.004598 (+ 3800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:32.005394 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:32.005405 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:32.005409 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:32.005580 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:32.005907 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:32.005912 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:32.005915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:32.006377 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:32.006689 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:32.006854 (+ 165us) server_negotiation.cc:299] Negotiation successful
0926 15:02:32.007090 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260926 15:02:32.008229 11175 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48930
W20260926 15:02:32.008895 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 11045: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 15:02:32.009092 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 11045
I20260926 15:02:32.027695 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 11184
Sep 26 15:02:32 dist-test-slave-7153 krb5kdc[10789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434950, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.123.67@KRBTEST.COM
I20260926 15:02:32.044507 11384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:32.028427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48192 (local address 127.9.123.67:41459)
0926 15:02:32.028594 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:32.028598 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:32.028617 (+ 19us) server_negotiation.cc:407] Connection header received
0926 15:02:32.028817 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:32.028821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:32.028889 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:32.028966 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:32.030611 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:32.031359 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:32.033166 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:32.033624 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:32.033718 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:32.038754 (+ 5036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:32.038768 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:32.038784 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:32.038829 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:32.041849 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:32.042660 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:32.042667 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:32.042670 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:32.042783 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:32.043149 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:32.043152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:32.043154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:32.043580 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:32.043797 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:32.044074 (+ 277us) server_negotiation.cc:299] Negotiation successful
0926 15:02:32.044240 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260926 15:02:32.045218 11314 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48192
W20260926 15:02:32.045866 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 11184: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 15:02:32.046000 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 11184
I20260926 15:02:32.060397 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 10802
I20260926 15:02:32.061471 10874 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:02:32.172785 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 10802
2026-09-26T15:02:32Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4860 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:02:32 dist-test-slave-7153 krb5kdc[11425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:02:32 dist-test-slave-7153 krb5kdc[11425](info): set up 2 sockets
Sep 26 15:02:32 dist-test-slave-7153 krb5kdc[11425](info): commencing operation
krb5kdc: starting...
W20260926 15:02:33.165081 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.954s user 0.000s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:02:33 dist-test-slave-7153 krb5kdc[11425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:02:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:02:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:02:33.347934 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:38981
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:46619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:38981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:33.675933 11438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:33.676301 11438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:33.676435 11438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:33.683970 11438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:02:33.684119 11438 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 15:02:33.684191 11438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:33.684284 11438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:33.684351 11438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:02:33.684433 11438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:02:33.694849 11438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:46619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:38981
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:38981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:33.696245 11438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:33.697724 11438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:33.709671 11446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:33.709710 11444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:33.709699 11443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:33.710426 11438 server_base.cc:1034] running on GCE node
I20260926 15:02:33.711321 11438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:33.712771 11438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:33.714026 11438 hybrid_clock.cc:648] HybridClock initialized: now 1790434953713955 us; error 48 us; skew 500 ppm
I20260926 15:02:33.714394 11438 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:33 dist-test-slave-7153 krb5kdc[11425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434953, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:33.717619 11438 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:02:33.719224 11438 webserver.cc:460] Webserver started at http://127.9.123.126:33257/ using document root <none> and password file <none>
I20260926 15:02:33.719985 11438 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:33.720096 11438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:33.720419 11438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:33.723639 11438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "ff517d6e5a6a4201a3b322b4fbe93c2c"
format_stamp: "Formatted at 2026-09-26 15:02:33 on dist-test-slave-7153"
server_key: "e29017a98b50670ba5a684b507ef1f28"
server_key_iv: "c6b5571173a91c7221ac4a0efc87ca77"
server_key_version: "encryptionkey@0"
I20260926 15:02:33.724471 11438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "ff517d6e5a6a4201a3b322b4fbe93c2c"
format_stamp: "Formatted at 2026-09-26 15:02:33 on dist-test-slave-7153"
server_key: "e29017a98b50670ba5a684b507ef1f28"
server_key_iv: "c6b5571173a91c7221ac4a0efc87ca77"
server_key_version: "encryptionkey@0"
I20260926 15:02:33.731688 11438 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260926 15:02:33.735673 11453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:33.737306 11438 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:02:33.737496 11438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "ff517d6e5a6a4201a3b322b4fbe93c2c"
format_stamp: "Formatted at 2026-09-26 15:02:33 on dist-test-slave-7153"
server_key: "e29017a98b50670ba5a684b507ef1f28"
server_key_iv: "c6b5571173a91c7221ac4a0efc87ca77"
server_key_version: "encryptionkey@0"
I20260926 15:02:33.737715 11438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:33.753290 11438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:33.757543 11438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:33.757797 11438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:33.777951 11438 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:38981
I20260926 15:02:33.777980 11514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:38981 every 8 connection(s)
I20260926 15:02:33.779328 11438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:02:33.783053 11515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:33.785485 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 11438
I20260926 15:02:33.785686 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:02:33.786494 9709 external_mini_cluster.cc:1428] Setting key c8ba3d83a17a4d218f8cae9f2dc53502
I20260926 15:02:33.795356 11515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c: Bootstrap starting.
I20260926 15:02:33.799213 11515 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:33.800518 11515 log.cc:824] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c: Log is configured to *not* fsync() on all Append() calls
Sep 26 15:02:33 dist-test-slave-7153 krb5kdc[11425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:33.803426 11515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c: No bootstrap required, opened a new log
I20260926 15:02:33.810317 11518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:33.788548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55546 (local address 127.9.123.126:38981)
0926 15:02:33.789361 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:33.789373 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:33.789447 (+ 74us) server_negotiation.cc:407] Connection header received
0926 15:02:33.791100 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:33.791123 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:33.791546 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:33.791913 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:33.793656 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:33.794986 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:33.796677 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:33.797497 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:33.797636 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:33.802080 (+ 4444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:33.802088 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:33.802105 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:33.802143 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:33.805683 (+ 3540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:33.806385 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:33.806395 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:33.806405 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:33.806629 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:33.806953 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:33.806958 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:33.806962 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:33.807692 (+ 730us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:33.807962 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:33.808329 (+ 367us) server_negotiation.cc:299] Negotiation successful
0926 15:02:33.808801 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":386,"threads_started":1}
I20260926 15:02:33.813542 11515 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff517d6e5a6a4201a3b322b4fbe93c2c" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 38981 } }
I20260926 15:02:33.813961 11515 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:33.814041 11515 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff517d6e5a6a4201a3b322b4fbe93c2c, State: Initialized, Role: FOLLOWER
I20260926 15:02:33.814553 11515 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff517d6e5a6a4201a3b322b4fbe93c2c" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 38981 } }
I20260926 15:02:33.814751 11515 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:02:33.814843 11515 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:02:33.814970 11515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:33.817137 11515 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff517d6e5a6a4201a3b322b4fbe93c2c" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 38981 } }
I20260926 15:02:33.817744 11515 leader_election.cc:302] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff517d6e5a6a4201a3b322b4fbe93c2c; no voters:
I20260926 15:02:33.818346 11515 leader_election.cc:288] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:02:33.818472 11520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:33.819134 11520 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c [term 1 LEADER]: Becoming Leader. State: Replica: ff517d6e5a6a4201a3b322b4fbe93c2c, State: Running, Role: LEADER
I20260926 15:02:33.819710 11520 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff517d6e5a6a4201a3b322b4fbe93c2c" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 38981 } }
I20260926 15:02:33.820302 11515 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:02:33.822461 11522 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff517d6e5a6a4201a3b322b4fbe93c2c. Latest consensus state: current_term: 1 leader_uuid: "ff517d6e5a6a4201a3b322b4fbe93c2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff517d6e5a6a4201a3b322b4fbe93c2c" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 38981 } } }
I20260926 15:02:33.822890 11522 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:33.822817 11521 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff517d6e5a6a4201a3b322b4fbe93c2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff517d6e5a6a4201a3b322b4fbe93c2c" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 38981 } } }
I20260926 15:02:33.823063 11521 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:33.824819 11527 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:02:33.830188 11527 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:02:33.839212 11527 catalog_manager.cc:1347] Generated new cluster ID: 9c1bdb0dd30841198755a027d854958e
I20260926 15:02:33.839342 11527 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:02:33.850229 11527 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:02:33.851643 11527 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:02:33.858419 11527 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c: Generated new TSK 0
I20260926 15:02:33.859274 11527 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.123.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
I20260926 15:02:33.917943 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:38981
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:46619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:34.267203 11541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:34.267508 11541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:34.267606 11541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:34.275101 11541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:34.275401 11541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:34.275554 11541 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 15:02:34.275624 11541 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 15:02:34.275727 11541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:02:34.286299 11541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:46619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.123.126:38981
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:34.287879 11541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:34.289376 11541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:34.302881 11549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:34.302886 11547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:34.302944 11546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:34.303934 11541 server_base.cc:1034] running on GCE node
I20260926 15:02:34.304582 11541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:34.305752 11541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:34.306986 11541 hybrid_clock.cc:648] HybridClock initialized: now 1790434954306929 us; error 57 us; skew 500 ppm
I20260926 15:02:34.307319 11541 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:34 dist-test-slave-7153 krb5kdc[11425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434954, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:34.316028 11541 init.cc:377] Logged in from keytab as kudu/127.9.123.65@KRBTEST.COM (short username kudu)
I20260926 15:02:34.317770 11541 webserver.cc:460] Webserver started at http://127.9.123.65:38465/ using document root <none> and password file <none>
I20260926 15:02:34.318493 11541 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:34.318627 11541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:34.318931 11541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:34.322005 11541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "649129489c4049588da477db3e4fba75"
format_stamp: "Formatted at 2026-09-26 15:02:34 on dist-test-slave-7153"
server_key: "f75aada7879fd6bb1c2862aa4a2341ca"
server_key_iv: "99a8bcd3242a675fa5f7da9d2c11d4b0"
server_key_version: "encryptionkey@0"
I20260926 15:02:34.322870 11541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "649129489c4049588da477db3e4fba75"
format_stamp: "Formatted at 2026-09-26 15:02:34 on dist-test-slave-7153"
server_key: "f75aada7879fd6bb1c2862aa4a2341ca"
server_key_iv: "99a8bcd3242a675fa5f7da9d2c11d4b0"
server_key_version: "encryptionkey@0"
I20260926 15:02:34.328462 11541 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260926 15:02:34.332182 11556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:34.333877 11541 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 15:02:34.334064 11541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "649129489c4049588da477db3e4fba75"
format_stamp: "Formatted at 2026-09-26 15:02:34 on dist-test-slave-7153"
server_key: "f75aada7879fd6bb1c2862aa4a2341ca"
server_key_iv: "99a8bcd3242a675fa5f7da9d2c11d4b0"
server_key_version: "encryptionkey@0"
I20260926 15:02:34.334283 11541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:34.355211 11541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:34.359614 11541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:34.359876 11541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:34.362573 11541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:34.362697 11541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:34.362820 11541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:34.362926 11541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 15:02:34 dist-test-slave-7153 krb5kdc[11425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434954, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:34.387972 11565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:34.363046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:39981 (local address 127.9.123.126:38981)
0926 15:02:34.363839 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:34.363843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:34.364961 (+ 1118us) server_negotiation.cc:407] Connection header received
0926 15:02:34.368043 (+ 3082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:34.368048 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:34.368137 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:34.368238 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:34.370579 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:34.371539 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:34.374020 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:34.374684 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:34.374842 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:34.380237 (+ 5395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:34.380245 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:34.380249 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:34.380294 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:34.383862 (+ 3568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:34.384985 (+ 1123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:34.384991 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:34.384993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:34.385170 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:34.385877 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:34.385882 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:34.385886 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:34.386189 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:34.386355 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:34.387486 (+ 1131us) server_negotiation.cc:299] Negotiation successful
0926 15:02:34.387678 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":607,"threads_started":1}
I20260926 15:02:34.389828 11566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:34.363912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:38981 (local address 127.9.123.65:39981)
0926 15:02:34.364783 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:34.364824 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:34.367673 (+ 2849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:34.368493 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:34.368505 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:34.369376 (+ 871us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:34.370278 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:34.370310 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:34.371728 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:34.371734 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:34.373828 (+ 2094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:34.373847 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:34.374976 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:34.374984 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:34.375263 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:34.376041 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:34.376094 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:34.379876 (+ 3782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:34.384094 (+ 4218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:34.384106 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:34.384132 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:34.384827 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:34.385316 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:34.385326 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:34.385329 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:34.385724 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:34.386352 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:34.386364 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:34.386970 (+ 606us) client_negotiation.cc:769] Sending connection context
0926 15:02:34.387358 (+ 388us) client_negotiation.cc:240] Negotiation successful
0926 15:02:34.388031 (+ 673us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":384,"threads_started":1}
I20260926 15:02:34.408938 11541 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:34481
I20260926 15:02:34.408962 11676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:34481 every 8 connection(s)
I20260926 15:02:34.410629 11541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:02:34.411187 11643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:34.396874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:38981 (local address 127.9.123.65:55711)
0926 15:02:34.398049 (+ 1175us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:34.398072 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:34.398255 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:34.398698 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:34.398703 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:34.399368 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:34.399820 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:34.399838 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:34.401037 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:34.401041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:34.402391 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:34.402410 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:34.403388 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:34.403392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:34.403542 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:34.404187 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:34.404213 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:34.405394 (+ 1181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:34.408565 (+ 3171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:34.408571 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:34.408576 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:34.409148 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:34.409579 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:34.409585 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:34.409589 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:34.409713 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:34.410290 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:34.410293 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:34.410447 (+ 154us) client_negotiation.cc:769] Sending connection context
0926 15:02:34.410553 (+ 106us) client_negotiation.cc:240] Negotiation successful
0926 15:02:34.410841 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":996,"thread_start_us":740,"threads_started":1}
I20260926 15:02:34.411259 11565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:34.396951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:55711 (local address 127.9.123.126:38981)
0926 15:02:34.397222 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:34.397231 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:34.398183 (+ 952us) server_negotiation.cc:407] Connection header received
0926 15:02:34.398413 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:34.398418 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:34.398528 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:34.398639 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:34.400033 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:34.400890 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:34.402553 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:34.403217 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:34.403360 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:34.405655 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:34.405660 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:34.405663 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:34.405698 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:34.408425 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:34.409285 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:34.409291 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:34.409294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:34.409452 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:34.409830 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:34.409834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:34.409838 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:34.410186 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:34.410346 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:34.410696 (+ 350us) server_negotiation.cc:299] Negotiation successful
0926 15:02:34.410918 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 15:02:34.413172 11677 heartbeater.cc:347] Connected to a master server at 127.9.123.126:38981
I20260926 15:02:34.414546 11677 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:34.415230 11677 heartbeater.cc:511] Master 127.9.123.126:38981 requested a full tablet report, sending...
I20260926 15:02:34.417286 11470 ts_manager.cc:194] Registered new tserver with Master: 649129489c4049588da477db3e4fba75 (127.9.123.65:34481)
I20260926 15:02:34.418771 11470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.65@KRBTEST.COM'} at 127.9.123.65:39981
I20260926 15:02:34.419270 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 11541
I20260926 15:02:34.419469 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:02:34.420096 9709 external_mini_cluster.cc:1428] Setting key dd70878dadb5fc913602488060096be0
WARNING: no policy specified for kudu/127.9.123.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.66@KRBTEST.COM" created.
I20260926 15:02:35.422147 11677 heartbeater.cc:503] Master 127.9.123.126:38981 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
W20260926 15:02:35.621884 9709 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.9.123.66: real 1.199s user 0.000s sys 0.008s
I20260926 15:02:35.622720 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:38981
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:46619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:35.954239 11681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:35.954592 11681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:35.954722 11681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:35.962050 11681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:35.962193 11681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:35.962316 11681 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 15:02:35.962385 11681 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 15:02:35.962504 11681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:02:35.972393 11681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:46619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.123.126:38981
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:35.973780 11681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:35.975134 11681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:35.987969 11686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:35.988042 11687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:35.987998 11689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:35.988615 11681 server_base.cc:1034] running on GCE node
I20260926 15:02:35.989351 11681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:35.990300 11681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:35.991539 11681 hybrid_clock.cc:648] HybridClock initialized: now 1790434955991478 us; error 46 us; skew 500 ppm
I20260926 15:02:35.991864 11681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:35 dist-test-slave-7153 krb5kdc[11425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434955, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:35.995131 11681 init.cc:377] Logged in from keytab as kudu/127.9.123.66@KRBTEST.COM (short username kudu)
I20260926 15:02:35.996675 11681 webserver.cc:460] Webserver started at http://127.9.123.66:41919/ using document root <none> and password file <none>
I20260926 15:02:35.997409 11681 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:35.997529 11681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:35.997820 11681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:36.000777 11681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a"
format_stamp: "Formatted at 2026-09-26 15:02:35 on dist-test-slave-7153"
server_key: "040a4d90ce05f31b081d9d9c25445d46"
server_key_iv: "e2fce2959b55bc63cd70f3fa57b76b7c"
server_key_version: "encryptionkey@0"
I20260926 15:02:36.001549 11681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a"
format_stamp: "Formatted at 2026-09-26 15:02:35 on dist-test-slave-7153"
server_key: "040a4d90ce05f31b081d9d9c25445d46"
server_key_iv: "e2fce2959b55bc63cd70f3fa57b76b7c"
server_key_version: "encryptionkey@0"
I20260926 15:02:36.006726 11681 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 15:02:36.010389 11696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:36.011895 11681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 15:02:36.012060 11681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a"
format_stamp: "Formatted at 2026-09-26 15:02:35 on dist-test-slave-7153"
server_key: "040a4d90ce05f31b081d9d9c25445d46"
server_key_iv: "e2fce2959b55bc63cd70f3fa57b76b7c"
server_key_version: "encryptionkey@0"
I20260926 15:02:36.012274 11681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:36.027948 11681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:36.032804 11681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:36.033063 11681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:36.035844 11681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:36.036026 11681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260926 15:02:36.036191 11681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:36.036306 11681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 15:02:36 dist-test-slave-7153 krb5kdc[11425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434955, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:36.063771 11705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:36.036669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:56343 (local address 127.9.123.126:38981)
0926 15:02:36.041539 (+ 4870us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:36.041546 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:36.042236 (+ 690us) server_negotiation.cc:407] Connection header received
0926 15:02:36.045540 (+ 3304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:36.045546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:36.045641 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:36.045758 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:36.048180 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:36.049196 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:36.051764 (+ 2568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:36.052436 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:36.052582 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:36.056699 (+ 4117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:36.056707 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:36.056712 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:36.056757 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:36.060151 (+ 3394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:36.061275 (+ 1124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:36.061282 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:36.061286 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:36.061412 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:36.061931 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:36.061935 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:36.061936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:36.062232 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:36.062351 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:36.063292 (+ 941us) server_negotiation.cc:299] Negotiation successful
0926 15:02:36.063492 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4726,"thread_start_us":4569,"threads_started":1}
I20260926 15:02:36.065850 11706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:36.037127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:38981 (local address 127.9.123.66:56343)
0926 15:02:36.041981 (+ 4854us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:36.042039 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:36.045062 (+ 3023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:36.046141 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:36.046159 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:36.046968 (+ 809us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:36.047913 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:36.047938 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:36.049368 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:36.049374 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:36.051571 (+ 2197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:36.051592 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:36.052720 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:36.052728 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:36.053017 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:36.053822 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:36.053845 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:36.056417 (+ 2572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:36.060300 (+ 3883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:36.060312 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:36.060336 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:36.061139 (+ 803us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:36.061518 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:36.061523 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:36.061527 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:36.061827 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:36.062348 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:36.062357 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:36.062886 (+ 529us) client_negotiation.cc:769] Sending connection context
0926 15:02:36.063228 (+ 342us) client_negotiation.cc:240] Negotiation successful
0926 15:02:36.063847 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4512,"thread_start_us":4282,"threads_started":1}
I20260926 15:02:36.084514 11681 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:45177
I20260926 15:02:36.084569 11816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:45177 every 8 connection(s)
I20260926 15:02:36.086256 11681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:02:36.088728 11817 heartbeater.cc:347] Connected to a master server at 127.9.123.126:38981
I20260926 15:02:36.088976 11817 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:36.089910 11817 heartbeater.cc:511] Master 127.9.123.126:38981 requested a full tablet report, sending...
I20260926 15:02:36.091010 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 11681
I20260926 15:02:36.091195 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:02:36.091711 9709 external_mini_cluster.cc:1428] Setting key 2e2067bae42fd9312237b7b60f6e776c
I20260926 15:02:36.091678 11470 ts_manager.cc:194] Registered new tserver with Master: 1fc5a333bcb14ce39f96fe2fef5c9f7a (127.9.123.66:45177)
I20260926 15:02:36.092777 11470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.66@KRBTEST.COM'} at 127.9.123.66:56343
I20260926 15:02:36.093086 11792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:36.075570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:38981 (local address 127.9.123.66:51647)
0926 15:02:36.076176 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:36.076195 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:36.076362 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:36.076782 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:36.076788 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:36.077659 (+ 871us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:36.078287 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:36.078311 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:36.080060 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:36.080067 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:36.082161 (+ 2094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:36.082188 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:36.083321 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:36.083329 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:36.083517 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:36.084340 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:36.084365 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:36.085530 (+ 1165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:36.089280 (+ 3750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:36.089286 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:36.089290 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:36.090716 (+ 1426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:36.091182 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:36.091187 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:36.091190 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:36.091343 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:36.091932 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:36.091936 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:36.092134 (+ 198us) client_negotiation.cc:769] Sending connection context
0926 15:02:36.092288 (+ 154us) client_negotiation.cc:240] Negotiation successful
0926 15:02:36.092646 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":335,"threads_started":1}
I20260926 15:02:36.093441 11705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:36.075542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:51647 (local address 127.9.123.126:38981)
0926 15:02:36.075719 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:36.075724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:36.076303 (+ 579us) server_negotiation.cc:407] Connection header received
0926 15:02:36.076499 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:36.076505 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:36.076591 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:36.076696 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:36.078567 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:36.079833 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:36.082578 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:36.083068 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:36.083189 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:36.085760 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:36.085766 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:36.085769 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:36.085803 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:36.089092 (+ 3289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:36.090850 (+ 1758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:36.090856 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:36.090860 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:36.091034 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:36.091458 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:36.091464 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:36.091468 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:36.091807 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:36.091994 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:36.092615 (+ 621us) server_negotiation.cc:299] Negotiation successful
0926 15:02:36.092868 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
WARNING: no policy specified for kudu/127.9.123.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.67@KRBTEST.COM" created.
I20260926 15:02:37.096366 11817 heartbeater.cc:503] Master 127.9.123.126:38981 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
W20260926 15:02:37.165508 9709 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.9.123.67: real 1.071s user 0.000s sys 0.008s
I20260926 15:02:37.166205 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:38981
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:46619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:37.505331 11821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:37.505683 11821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:37.505827 11821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:37.513923 11821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:37.514068 11821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:37.514189 11821 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260926 15:02:37.514254 11821 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 15:02:37.514360 11821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:02:37.526484 11821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:46619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.123.126:38981
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:37.528134 11821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:37.529647 11821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:37.543514 11829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:37.543571 11826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:37.543551 11827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:37.544162 11821 server_base.cc:1034] running on GCE node
I20260926 15:02:37.545531 11821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:37.546626 11821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:37.547858 11821 hybrid_clock.cc:648] HybridClock initialized: now 1790434957547802 us; error 54 us; skew 500 ppm
I20260926 15:02:37.548211 11821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:37 dist-test-slave-7153 krb5kdc[11425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434957, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:37.551406 11821 init.cc:377] Logged in from keytab as kudu/127.9.123.67@KRBTEST.COM (short username kudu)
I20260926 15:02:37.552958 11821 webserver.cc:460] Webserver started at http://127.9.123.67:39429/ using document root <none> and password file <none>
I20260926 15:02:37.553871 11821 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:37.554013 11821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:37.554299 11821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:37.557521 11821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "a43d1b2c6d624cf698b6cd91b0e68335"
format_stamp: "Formatted at 2026-09-26 15:02:37 on dist-test-slave-7153"
server_key: "854c358d354dae6bd1b7f4ba5dd80d04"
server_key_iv: "13c79e401ae8026ffdbaa8f15a26b400"
server_key_version: "encryptionkey@0"
I20260926 15:02:37.558269 11821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "a43d1b2c6d624cf698b6cd91b0e68335"
format_stamp: "Formatted at 2026-09-26 15:02:37 on dist-test-slave-7153"
server_key: "854c358d354dae6bd1b7f4ba5dd80d04"
server_key_iv: "13c79e401ae8026ffdbaa8f15a26b400"
server_key_version: "encryptionkey@0"
I20260926 15:02:37.563897 11821 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 15:02:37.567663 11836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:37.569236 11821 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:02:37.569393 11821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "a43d1b2c6d624cf698b6cd91b0e68335"
format_stamp: "Formatted at 2026-09-26 15:02:37 on dist-test-slave-7153"
server_key: "854c358d354dae6bd1b7f4ba5dd80d04"
server_key_iv: "13c79e401ae8026ffdbaa8f15a26b400"
server_key_version: "encryptionkey@0"
I20260926 15:02:37.569597 11821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:37.602239 11821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:37.606455 11821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:37.606732 11821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:37.609375 11821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:37.609472 11821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:37.609592 11821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:37.609661 11821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 15:02:37 dist-test-slave-7153 krb5kdc[11425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434957, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:37.634187 11845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:37.610126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:60201 (local address 127.9.123.126:38981)
0926 15:02:37.610545 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:37.610576 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:37.611826 (+ 1250us) server_negotiation.cc:407] Connection header received
0926 15:02:37.614801 (+ 2975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:37.614808 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:37.614901 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:37.615013 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:37.617197 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:37.618230 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:37.620653 (+ 2423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:37.621298 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:37.621416 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:37.626308 (+ 4892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:37.626316 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:37.626322 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:37.626385 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:37.629973 (+ 3588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:37.631097 (+ 1124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:37.631104 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:37.631107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:37.631283 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:37.631876 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:37.631881 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:37.631886 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:37.632286 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:37.632425 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:37.633551 (+ 1126us) server_negotiation.cc:299] Negotiation successful
0926 15:02:37.633785 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":198,"threads_started":1}
I20260926 15:02:37.635299 11846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:37.610804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:38981 (local address 127.9.123.67:60201)
0926 15:02:37.611630 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:37.611678 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:37.614406 (+ 2728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:37.615342 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:37.615356 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:37.616053 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:37.616966 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:37.616993 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:37.618409 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:37.618415 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:37.620479 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:37.620502 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:37.621554 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:37.621562 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:37.621912 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:37.622720 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:37.622759 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:37.625950 (+ 3191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:37.630223 (+ 4273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:37.630234 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:37.630259 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:37.630913 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:37.631417 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:37.631422 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:37.631426 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:37.631756 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:37.632427 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:37.632437 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:37.633015 (+ 578us) client_negotiation.cc:769] Sending connection context
0926 15:02:37.633422 (+ 407us) client_negotiation.cc:240] Negotiation successful
0926 15:02:37.634035 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":314,"threads_started":1}
I20260926 15:02:37.654134 11821 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:42333
I20260926 15:02:37.654182 11956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:42333 every 8 connection(s)
I20260926 15:02:37.655707 11821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:02:37.656483 11845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:37.641872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:55555 (local address 127.9.123.126:38981)
0926 15:02:37.642077 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:37.642083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:37.642908 (+ 825us) server_negotiation.cc:407] Connection header received
0926 15:02:37.643138 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:37.643144 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:37.643216 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:37.643297 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:37.644431 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:37.645302 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:37.647423 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:37.647937 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:37.648087 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:37.650597 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:37.650602 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:37.650605 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:37.650656 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:37.653681 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:37.654407 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:37.654412 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:37.654416 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:37.654554 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:37.654983 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:37.654988 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:37.654991 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:37.655360 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:37.655516 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:37.655898 (+ 382us) server_negotiation.cc:299] Negotiation successful
0926 15:02:37.656136 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 15:02:37.656611 11930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:37.642219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:38981 (local address 127.9.123.67:55555)
0926 15:02:37.642794 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:37.642810 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:37.642978 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:37.643338 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:37.643342 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:37.643983 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:37.644285 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:37.644296 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:37.645459 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:37.645467 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:37.647245 (+ 1778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:37.647264 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:37.648317 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:37.648323 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:37.648539 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:37.649196 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:37.649222 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:37.650388 (+ 1166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:37.653808 (+ 3420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:37.653813 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:37.653818 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:37.654294 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:37.654693 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:37.654697 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:37.654700 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:37.654842 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:37.655479 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:37.655482 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:37.655652 (+ 170us) client_negotiation.cc:769] Sending connection context
0926 15:02:37.655787 (+ 135us) client_negotiation.cc:240] Negotiation successful
0926 15:02:37.656094 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":325,"threads_started":1}
I20260926 15:02:37.658886 11957 heartbeater.cc:347] Connected to a master server at 127.9.123.126:38981
I20260926 15:02:37.659108 11957 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:37.659782 11957 heartbeater.cc:511] Master 127.9.123.126:38981 requested a full tablet report, sending...
I20260926 15:02:37.661036 11470 ts_manager.cc:194] Registered new tserver with Master: a43d1b2c6d624cf698b6cd91b0e68335 (127.9.123.67:42333)
I20260926 15:02:37.661864 11470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.67@KRBTEST.COM'} at 127.9.123.67:60201
I20260926 15:02:37.664131 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 11821
I20260926 15:02:37.664281 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:02:37.664719 9709 external_mini_cluster.cc:1428] Setting key af661fa71f678441fb9dde9077f2272e
I20260926 15:02:37.667510 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 15:02:37 dist-test-slave-7153 krb5kdc[11425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 15:02:37 dist-test-slave-7153 krb5kdc[11425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:37.787374 11845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:37.772576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55552 (local address 127.9.123.126:38981)
0926 15:02:37.772773 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:37.772778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:37.772797 (+ 19us) server_negotiation.cc:407] Connection header received
0926 15:02:37.772850 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:37.772854 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:37.772931 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:37.773044 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:37.774579 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:37.775322 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:37.776595 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:37.777017 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:37.777157 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:37.781739 (+ 4582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:37.781744 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:37.781748 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:37.781785 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:37.784517 (+ 2732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:37.785379 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:37.785384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:37.785386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:37.785539 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:37.786021 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:37.786026 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:37.786029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:37.786360 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:37.786574 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:37.786917 (+ 343us) server_negotiation.cc:299] Negotiation successful
0926 15:02:37.787099 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260926 15:02:37.791527 11470 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:02:37.798514 11470 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:02:37.831512 11970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:37.811929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:45177 (local address 127.0.0.1:47604)
0926 15:02:37.813119 (+ 1190us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:37.813161 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:37.813356 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:37.821784 (+ 8428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:37.821793 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:37.821836 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:37.822306 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:37.822319 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:37.825168 (+ 2849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:37.825176 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:37.827932 (+ 2756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:37.827954 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:37.830110 (+ 2156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:37.830118 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:37.830459 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:37.830574 (+ 115us) client_negotiation.cc:769] Sending connection context
0926 15:02:37.830782 (+ 208us) client_negotiation.cc:240] Negotiation successful
0926 15:02:37.831111 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":919,"thread_start_us":809,"threads_started":1}
I20260926 15:02:37.831579 11971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:37.812085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47604 (local address 127.9.123.66:45177)
0926 15:02:37.812709 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:37.812719 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:37.820132 (+ 7413us) server_negotiation.cc:407] Connection header received
0926 15:02:37.820283 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:37.820294 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:37.820645 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:37.822789 (+ 2144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:37.823042 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:37.824950 (+ 1908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:37.828257 (+ 3307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:37.829886 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:37.830000 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:37.830234 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:37.831105 (+ 871us) server_negotiation.cc:299] Negotiation successful
0926 15:02:37.831286 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":339,"threads_started":1}
I20260926 15:02:37.836081 11747 tablet_service.cc:1467] Processing CreateTablet for tablet 5b9cd727ab3a481e9c3a386908f33f49 (DEFAULT_TABLE table=test-table [id=0e4041b7e6a241068e6d0fda3b465c50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:37.837462 11974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:37.812644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:42333 (local address 127.0.0.1:39884)
0926 15:02:37.829332 (+ 16688us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:37.829355 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:37.829510 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:37.831054 (+ 1544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:37.831058 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:37.831082 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:37.831446 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:37.831458 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:37.833774 (+ 2316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:37.833779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:37.835333 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:37.835344 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:37.836624 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:37.836631 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:37.836801 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:37.836830 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 15:02:37.836927 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 15:02:37.837060 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16497,"thread_start_us":16665,"threads_started":1}
I20260926 15:02:37.837765 11973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:37.812779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39884 (local address 127.9.123.67:42333)
0926 15:02:37.828576 (+ 15797us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:37.828585 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:37.829575 (+ 990us) server_negotiation.cc:407] Connection header received
0926 15:02:37.829700 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:37.829709 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:37.830006 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:37.830748 (+ 742us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:37.831817 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:37.833610 (+ 1793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:37.835489 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:37.836405 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:37.836515 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:37.836727 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:37.837045 (+ 318us) server_negotiation.cc:299] Negotiation successful
0926 15:02:37.837230 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15594,"thread_start_us":15784,"threads_started":1}
I20260926 15:02:37.837865 11747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b9cd727ab3a481e9c3a386908f33f49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:37.838899 11975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:37.814129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50718 (local address 127.9.123.65:34481)
0926 15:02:37.822544 (+ 8415us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:37.822556 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:37.829727 (+ 7171us) server_negotiation.cc:407] Connection header received
0926 15:02:37.829860 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:37.829867 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:37.830165 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:37.830357 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:37.830980 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:37.832353 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:37.834717 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:37.836142 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:37.836274 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:37.836479 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:37.837496 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0926 15:02:37.837660 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8213,"thread_start_us":8305,"threads_started":1}
I20260926 15:02:37.839313 11972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:37.813616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:34481 (local address 127.0.0.1:50718)
0926 15:02:37.829587 (+ 15971us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:37.829606 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:37.829729 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:37.830346 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:37.830351 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:37.830380 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:37.830801 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:37.830813 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:37.832502 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:37.832506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:37.834534 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:37.834549 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:37.836713 (+ 2164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:37.836722 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:37.836890 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:37.836921 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 15:02:37.837005 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 15:02:37.837169 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15843,"mutex_wait_us":1,"thread_start_us":15348,"threads_started":1}
I20260926 15:02:37.841333 11887 tablet_service.cc:1467] Processing CreateTablet for tablet 5b9cd727ab3a481e9c3a386908f33f49 (DEFAULT_TABLE table=test-table [id=0e4041b7e6a241068e6d0fda3b465c50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:37.842864 11887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b9cd727ab3a481e9c3a386908f33f49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:37.843060 11607 tablet_service.cc:1467] Processing CreateTablet for tablet 5b9cd727ab3a481e9c3a386908f33f49 (DEFAULT_TABLE table=test-table [id=0e4041b7e6a241068e6d0fda3b465c50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:37.844396 11607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b9cd727ab3a481e9c3a386908f33f49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:37.849483 11976 tablet_bootstrap.cc:492] T 5b9cd727ab3a481e9c3a386908f33f49 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Bootstrap starting.
I20260926 15:02:37.852900 11976 tablet_bootstrap.cc:654] T 5b9cd727ab3a481e9c3a386908f33f49 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:37.854121 11976 log.cc:824] T 5b9cd727ab3a481e9c3a386908f33f49 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:37.859431 11976 tablet_bootstrap.cc:492] T 5b9cd727ab3a481e9c3a386908f33f49 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: No bootstrap required, opened a new log
I20260926 15:02:37.859730 11976 ts_tablet_manager.cc:1397] T 5b9cd727ab3a481e9c3a386908f33f49 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260926 15:02:37.860569 11978 tablet_bootstrap.cc:492] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75: Bootstrap starting.
I20260926 15:02:37.861562 11979 tablet_bootstrap.cc:492] T 5b9cd727ab3a481e9c3a386908f33f49 P a43d1b2c6d624cf698b6cd91b0e68335: Bootstrap starting.
I20260926 15:02:37.864524 11978 tablet_bootstrap.cc:654] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:37.865010 11979 tablet_bootstrap.cc:654] T 5b9cd727ab3a481e9c3a386908f33f49 P a43d1b2c6d624cf698b6cd91b0e68335: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:37.865626 11978 log.cc:824] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:37.866195 11979 log.cc:824] T 5b9cd727ab3a481e9c3a386908f33f49 P a43d1b2c6d624cf698b6cd91b0e68335: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:37.868487 11978 tablet_bootstrap.cc:492] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75: No bootstrap required, opened a new log
I20260926 15:02:37.868772 11978 ts_tablet_manager.cc:1397] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260926 15:02:37.870349 11979 tablet_bootstrap.cc:492] T 5b9cd727ab3a481e9c3a386908f33f49 P a43d1b2c6d624cf698b6cd91b0e68335: No bootstrap required, opened a new log
I20260926 15:02:37.870285 11976 raft_consensus.cc:348] T 5b9cd727ab3a481e9c3a386908f33f49 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:37.870651 11979 ts_tablet_manager.cc:1397] T 5b9cd727ab3a481e9c3a386908f33f49 P a43d1b2c6d624cf698b6cd91b0e68335: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260926 15:02:37.870682 11976 raft_consensus.cc:374] T 5b9cd727ab3a481e9c3a386908f33f49 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:37.870771 11976 raft_consensus.cc:729] T 5b9cd727ab3a481e9c3a386908f33f49 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc5a333bcb14ce39f96fe2fef5c9f7a, State: Initialized, Role: FOLLOWER
I20260926 15:02:37.871359 11976 consensus_queue.cc:260] T 5b9cd727ab3a481e9c3a386908f33f49 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:37.873142 11976 ts_tablet_manager.cc:1428] T 5b9cd727ab3a481e9c3a386908f33f49 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Time spent starting tablet: real 0.013s user 0.006s sys 0.008s
I20260926 15:02:37.879599 11978 raft_consensus.cc:348] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:37.880128 11978 raft_consensus.cc:374] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:37.880228 11978 raft_consensus.cc:729] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 649129489c4049588da477db3e4fba75, State: Initialized, Role: FOLLOWER
I20260926 15:02:37.880920 11978 consensus_queue.cc:260] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:37.881572 11979 raft_consensus.cc:348] T 5b9cd727ab3a481e9c3a386908f33f49 P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:37.882021 11979 raft_consensus.cc:374] T 5b9cd727ab3a481e9c3a386908f33f49 P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:37.882117 11979 raft_consensus.cc:729] T 5b9cd727ab3a481e9c3a386908f33f49 P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a43d1b2c6d624cf698b6cd91b0e68335, State: Initialized, Role: FOLLOWER
I20260926 15:02:37.882836 11978 ts_tablet_manager.cc:1428] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260926 15:02:37.882748 11979 consensus_queue.cc:260] T 5b9cd727ab3a481e9c3a386908f33f49 P a43d1b2c6d624cf698b6cd91b0e68335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:37.883786 11957 heartbeater.cc:503] Master 127.9.123.126:38981 was elected leader, sending a full tablet report...
I20260926 15:02:37.884454 11979 ts_tablet_manager.cc:1428] T 5b9cd727ab3a481e9c3a386908f33f49 P a43d1b2c6d624cf698b6cd91b0e68335: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260926 15:02:37.907842 11958 tablet.cc:2369] T 5b9cd727ab3a481e9c3a386908f33f49 P a43d1b2c6d624cf698b6cd91b0e68335: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 15:02:37.916599 11678 tablet.cc:2369] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 15:02:38.090811 11818 tablet.cc:2369] T 5b9cd727ab3a481e9c3a386908f33f49 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:02:38.109714 11983 raft_consensus.cc:482] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:38.110019 11983 raft_consensus.cc:504] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:38.111755 11983 leader_election.cc:288] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc5a333bcb14ce39f96fe2fef5c9f7a (127.9.123.66:45177), a43d1b2c6d624cf698b6cd91b0e68335 (127.9.123.67:42333)
I20260926 15:02:38.119356 11971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:38.112710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:44007 (local address 127.9.123.66:45177)
0926 15:02:38.112913 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:38.112919 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:38.113364 (+ 445us) server_negotiation.cc:407] Connection header received
0926 15:02:38.113660 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:38.113666 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:38.113733 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:38.113820 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:38.114368 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.115117 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.117192 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.118277 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.118397 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:38.118463 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:38.118997 (+ 534us) server_negotiation.cc:299] Negotiation successful
0926 15:02:38.119125 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 15:02:38.119517 11985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:38.112171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:45177 (local address 127.9.123.65:44007)
0926 15:02:38.113263 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:38.113280 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:38.113509 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:38.113850 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:38.113854 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:38.113875 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:38.114239 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:38.114247 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.115273 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.115278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:38.116952 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:38.116968 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.118644 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.118649 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:38.118832 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:38.118867 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 15:02:38.118959 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 15:02:38.119105 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"mutex_wait_us":1,"thread_start_us":1109,"threads_started":1}
I20260926 15:02:38.120301 11973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:38.113228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:47719 (local address 127.9.123.67:42333)
0926 15:02:38.113420 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:38.113425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:38.113442 (+ 17us) server_negotiation.cc:407] Connection header received
0926 15:02:38.113510 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:38.113515 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:38.113593 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:38.113679 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:38.114368 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.115368 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.117968 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.119123 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.119264 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:38.119339 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:38.119934 (+ 595us) server_negotiation.cc:299] Negotiation successful
0926 15:02:38.120075 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 15:02:38.120359 11986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:38.112209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:42333 (local address 127.9.123.65:47719)
0926 15:02:38.112982 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:38.113017 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:38.113179 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:38.113704 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:38.113707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:38.113731 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:38.114142 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:38.114157 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.115579 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.115584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:38.117776 (+ 2192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:38.117793 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.119577 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.119584 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:38.119748 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:38.119781 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 15:02:38.119867 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 15:02:38.120037 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":478,"threads_started":1}
I20260926 15:02:38.120719 11767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b9cd727ab3a481e9c3a386908f33f49" candidate_uuid: "649129489c4049588da477db3e4fba75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" is_pre_election: true
I20260926 15:02:38.121258 11767 raft_consensus.cc:2454] T 5b9cd727ab3a481e9c3a386908f33f49 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 649129489c4049588da477db3e4fba75 in term 0.
I20260926 15:02:38.121984 11559 leader_election.cc:302] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc5a333bcb14ce39f96fe2fef5c9f7a, 649129489c4049588da477db3e4fba75; no voters:
I20260926 15:02:38.122018 11907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b9cd727ab3a481e9c3a386908f33f49" candidate_uuid: "649129489c4049588da477db3e4fba75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" is_pre_election: true
I20260926 15:02:38.122434 11983 raft_consensus.cc:2781] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:38.122506 11907 raft_consensus.cc:2454] T 5b9cd727ab3a481e9c3a386908f33f49 P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 649129489c4049588da477db3e4fba75 in term 0.
I20260926 15:02:38.122583 11983 raft_consensus.cc:482] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:38.122666 11983 raft_consensus.cc:3037] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:38.176393 11983 raft_consensus.cc:504] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:38.177341 11983 leader_election.cc:288] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75 [CANDIDATE]: Term 1 election: Requested vote from peers 1fc5a333bcb14ce39f96fe2fef5c9f7a (127.9.123.66:45177), a43d1b2c6d624cf698b6cd91b0e68335 (127.9.123.67:42333)
I20260926 15:02:38.177996 11767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b9cd727ab3a481e9c3a386908f33f49" candidate_uuid: "649129489c4049588da477db3e4fba75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a"
I20260926 15:02:38.177974 11907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b9cd727ab3a481e9c3a386908f33f49" candidate_uuid: "649129489c4049588da477db3e4fba75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a43d1b2c6d624cf698b6cd91b0e68335"
I20260926 15:02:38.178225 11767 raft_consensus.cc:3037] T 5b9cd727ab3a481e9c3a386908f33f49 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:38.178225 11907 raft_consensus.cc:3037] T 5b9cd727ab3a481e9c3a386908f33f49 P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:38.283700 11767 raft_consensus.cc:2454] T 5b9cd727ab3a481e9c3a386908f33f49 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 649129489c4049588da477db3e4fba75 in term 1.
I20260926 15:02:38.283700 11907 raft_consensus.cc:2454] T 5b9cd727ab3a481e9c3a386908f33f49 P a43d1b2c6d624cf698b6cd91b0e68335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 649129489c4049588da477db3e4fba75 in term 1.
I20260926 15:02:38.284327 11559 leader_election.cc:302] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc5a333bcb14ce39f96fe2fef5c9f7a, 649129489c4049588da477db3e4fba75; no voters:
I20260926 15:02:38.284724 11983 raft_consensus.cc:2781] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:38.285240 11983 raft_consensus.cc:686] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75 [term 1 LEADER]: Becoming Leader. State: Replica: 649129489c4049588da477db3e4fba75, State: Running, Role: LEADER
I20260926 15:02:38.285805 11983 consensus_queue.cc:237] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:38.291494 11469 catalog_manager.cc:5495] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 649129489c4049588da477db3e4fba75 (127.9.123.65). New cstate: current_term: 1 leader_uuid: "649129489c4049588da477db3e4fba75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } health_report { overall_health: UNKNOWN } } }
Sep 26 15:02:38 dist-test-slave-7153 krb5kdc[11425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434953, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:38.440969 11995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:38.427352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:38981 (local address 127.0.0.1:55566)
0926 15:02:38.428066 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:38.428085 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:38.428242 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:38.428534 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:38.428539 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:38.429313 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:38.429559 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:38.429570 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.430625 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.430630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:38.431588 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:38.431599 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.432305 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.432311 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:38.432479 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:38.433181 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:38.433197 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:38.435390 (+ 2193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:38.438350 (+ 2960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:38.438366 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:38.438376 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:38.438841 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:38.439200 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:38.439205 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:38.439208 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:38.439345 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:38.439851 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:38.439860 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:38.440009 (+ 149us) client_negotiation.cc:769] Sending connection context
0926 15:02:38.440167 (+ 158us) client_negotiation.cc:240] Negotiation successful
0926 15:02:38.440481 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"mutex_wait_us":2,"thread_start_us":326,"threads_started":1}
I20260926 15:02:38.440974 11994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:38.427352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55566 (local address 127.9.123.126:38981)
0926 15:02:38.428061 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:38.428067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:38.428174 (+ 107us) server_negotiation.cc:407] Connection header received
0926 15:02:38.428352 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:38.428357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:38.428416 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:38.428498 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:38.429737 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.430483 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.431728 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.432138 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.432243 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:38.435588 (+ 3345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:38.435593 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:38.435596 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:38.435625 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:38.438224 (+ 2599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:38.438949 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:38.438954 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:38.438958 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:38.439083 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:38.439439 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:38.439444 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:38.439448 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:38.439724 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:38.439852 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:38.440260 (+ 408us) server_negotiation.cc:299] Negotiation successful
0926 15:02:38.440614 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":153,"server-negotiator.queue_time_us":600,"thread_start_us":368,"threads_started":1}
I20260926 15:02:38.446445 11469 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:55566:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 15:02:38.447520 11469 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:02:38.454238 11887 tablet_service.cc:1467] Processing CreateTablet for tablet cf54b2c6363d49f9b0b93fc33fcc5208 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ef0b1dcb035d4a33b66e568772090c0a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 15:02:38.454010 11747 tablet_service.cc:1467] Processing CreateTablet for tablet cf54b2c6363d49f9b0b93fc33fcc5208 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ef0b1dcb035d4a33b66e568772090c0a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 15:02:38.453953 11607 tablet_service.cc:1467] Processing CreateTablet for tablet cf54b2c6363d49f9b0b93fc33fcc5208 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ef0b1dcb035d4a33b66e568772090c0a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 15:02:38.454957 11747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf54b2c6363d49f9b0b93fc33fcc5208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:38.454968 11887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf54b2c6363d49f9b0b93fc33fcc5208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:38.454962 11607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf54b2c6363d49f9b0b93fc33fcc5208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:38.461045 11996 tablet_bootstrap.cc:492] T cf54b2c6363d49f9b0b93fc33fcc5208 P 649129489c4049588da477db3e4fba75: Bootstrap starting.
I20260926 15:02:38.462419 11996 tablet_bootstrap.cc:654] T cf54b2c6363d49f9b0b93fc33fcc5208 P 649129489c4049588da477db3e4fba75: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:38.465013 11998 tablet_bootstrap.cc:492] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Bootstrap starting.
I20260926 15:02:38.465333 11996 tablet_bootstrap.cc:492] T cf54b2c6363d49f9b0b93fc33fcc5208 P 649129489c4049588da477db3e4fba75: No bootstrap required, opened a new log
I20260926 15:02:38.465488 11996 ts_tablet_manager.cc:1397] T cf54b2c6363d49f9b0b93fc33fcc5208 P 649129489c4049588da477db3e4fba75: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260926 15:02:38.466130 11996 raft_consensus.cc:348] T cf54b2c6363d49f9b0b93fc33fcc5208 P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:38.466326 11996 raft_consensus.cc:374] T cf54b2c6363d49f9b0b93fc33fcc5208 P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:38.466367 11998 tablet_bootstrap.cc:654] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:38.466408 11996 raft_consensus.cc:729] T cf54b2c6363d49f9b0b93fc33fcc5208 P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 649129489c4049588da477db3e4fba75, State: Initialized, Role: FOLLOWER
I20260926 15:02:38.466620 11996 consensus_queue.cc:260] T cf54b2c6363d49f9b0b93fc33fcc5208 P 649129489c4049588da477db3e4fba75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:38.467113 11996 ts_tablet_manager.cc:1428] T cf54b2c6363d49f9b0b93fc33fcc5208 P 649129489c4049588da477db3e4fba75: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260926 15:02:38.467053 11983 tablet_replica.cc:439] T cf54b2c6363d49f9b0b93fc33fcc5208 P 649129489c4049588da477db3e4fba75: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } } }
I20260926 15:02:38.467373 11983 tablet_replica.cc:442] T cf54b2c6363d49f9b0b93fc33fcc5208 P 649129489c4049588da477db3e4fba75: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 15:02:38.468116 11997 tablet_bootstrap.cc:492] T cf54b2c6363d49f9b0b93fc33fcc5208 P a43d1b2c6d624cf698b6cd91b0e68335: Bootstrap starting.
I20260926 15:02:38.469189 11998 tablet_bootstrap.cc:492] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: No bootstrap required, opened a new log
I20260926 15:02:38.469342 11998 ts_tablet_manager.cc:1397] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260926 15:02:38.469692 11997 tablet_bootstrap.cc:654] T cf54b2c6363d49f9b0b93fc33fcc5208 P a43d1b2c6d624cf698b6cd91b0e68335: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:38.470096 11998 raft_consensus.cc:348] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:38.470297 11998 raft_consensus.cc:374] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:38.470389 11998 raft_consensus.cc:729] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc5a333bcb14ce39f96fe2fef5c9f7a, State: Initialized, Role: FOLLOWER
I20260926 15:02:38.470628 11998 consensus_queue.cc:260] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:38.471508 11998 ts_tablet_manager.cc:1428] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 15:02:38.472061 12002 tablet_replica.cc:439] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } } }
I20260926 15:02:38.472443 12002 tablet_replica.cc:442] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 15:02:38.473340 11997 tablet_bootstrap.cc:492] T cf54b2c6363d49f9b0b93fc33fcc5208 P a43d1b2c6d624cf698b6cd91b0e68335: No bootstrap required, opened a new log
I20260926 15:02:38.473501 11997 ts_tablet_manager.cc:1397] T cf54b2c6363d49f9b0b93fc33fcc5208 P a43d1b2c6d624cf698b6cd91b0e68335: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.004s
I20260926 15:02:38.474313 11997 raft_consensus.cc:348] T cf54b2c6363d49f9b0b93fc33fcc5208 P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:38.474493 11997 raft_consensus.cc:374] T cf54b2c6363d49f9b0b93fc33fcc5208 P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:38.474571 11997 raft_consensus.cc:729] T cf54b2c6363d49f9b0b93fc33fcc5208 P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a43d1b2c6d624cf698b6cd91b0e68335, State: Initialized, Role: FOLLOWER
I20260926 15:02:38.474776 11997 consensus_queue.cc:260] T cf54b2c6363d49f9b0b93fc33fcc5208 P a43d1b2c6d624cf698b6cd91b0e68335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:38.475445 11997 ts_tablet_manager.cc:1428] T cf54b2c6363d49f9b0b93fc33fcc5208 P a43d1b2c6d624cf698b6cd91b0e68335: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 15:02:38.475534 12003 tablet_replica.cc:439] T cf54b2c6363d49f9b0b93fc33fcc5208 P a43d1b2c6d624cf698b6cd91b0e68335: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } } }
I20260926 15:02:38.475879 12003 tablet_replica.cc:442] T cf54b2c6363d49f9b0b93fc33fcc5208 P a43d1b2c6d624cf698b6cd91b0e68335: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 15:02:38.535269 12002 raft_consensus.cc:482] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:38.535465 12002 raft_consensus.cc:504] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:38.536886 12002 leader_election.cc:288] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 649129489c4049588da477db3e4fba75 (127.9.123.65:34481), a43d1b2c6d624cf698b6cd91b0e68335 (127.9.123.67:42333)
I20260926 15:02:38.544088 11973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:38.537299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:37417 (local address 127.9.123.67:42333)
0926 15:02:38.537492 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:38.537497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:38.538187 (+ 690us) server_negotiation.cc:407] Connection header received
0926 15:02:38.538528 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:38.538535 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:38.538650 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:38.538705 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:38.539427 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.540248 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.542250 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.543153 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.543279 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:38.543362 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:38.543672 (+ 310us) server_negotiation.cc:299] Negotiation successful
0926 15:02:38.543803 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 15:02:38.544184 12005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:38.537359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:59465 (local address 127.9.123.65:34481)
0926 15:02:38.538353 (+ 994us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:38.538359 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:38.538694 (+ 335us) server_negotiation.cc:407] Connection header received
0926 15:02:38.538846 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:38.538851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:38.538923 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:38.539004 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:38.539682 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.540489 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.542377 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.543192 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.543358 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:38.543429 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:38.543724 (+ 295us) server_negotiation.cc:299] Negotiation successful
0926 15:02:38.543851 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":870,"thread_start_us":726,"threads_started":1}
I20260926 15:02:38.544184 12004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:38.537292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:42333 (local address 127.9.123.66:37417)
0926 15:02:38.538061 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:38.538083 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:38.538326 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:38.538881 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:38.538884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:38.538905 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:38.539260 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:38.539277 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.540386 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.540391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:38.542117 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:38.542131 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.543346 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.543352 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:38.543531 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:38.543554 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 15:02:38.543628 (+ 74us) client_negotiation.cc:240] Negotiation successful
0926 15:02:38.543753 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":406,"threads_started":1}
I20260926 15:02:38.544591 12006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:38.537319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:34481 (local address 127.9.123.66:59465)
0926 15:02:38.538597 (+ 1278us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:38.538611 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:38.538736 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:38.539045 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:38.539049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:38.539073 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:38.539504 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:38.539516 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.540665 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.540669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:38.542236 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:38.542248 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.543428 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.543434 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:38.543578 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:38.543600 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 15:02:38.543689 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 15:02:38.543803 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1162,"thread_start_us":987,"threads_started":1}
I20260926 15:02:38.544778 11907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf54b2c6363d49f9b0b93fc33fcc5208" candidate_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" is_pre_election: true
I20260926 15:02:38.545042 11907 raft_consensus.cc:2454] T cf54b2c6363d49f9b0b93fc33fcc5208 P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc5a333bcb14ce39f96fe2fef5c9f7a in term 0.
I20260926 15:02:38.545497 11627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf54b2c6363d49f9b0b93fc33fcc5208" candidate_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "649129489c4049588da477db3e4fba75" is_pre_election: true
I20260926 15:02:38.545688 11698 leader_election.cc:302] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc5a333bcb14ce39f96fe2fef5c9f7a, a43d1b2c6d624cf698b6cd91b0e68335; no voters:
I20260926 15:02:38.545917 11627 raft_consensus.cc:2454] T cf54b2c6363d49f9b0b93fc33fcc5208 P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc5a333bcb14ce39f96fe2fef5c9f7a in term 0.
I20260926 15:02:38.546079 12002 raft_consensus.cc:2781] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:38.546197 12002 raft_consensus.cc:482] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:38.546303 12002 raft_consensus.cc:3037] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:38.588904 12002 raft_consensus.cc:504] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:38.589715 12002 leader_election.cc:288] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [CANDIDATE]: Term 1 election: Requested vote from peers 649129489c4049588da477db3e4fba75 (127.9.123.65:34481), a43d1b2c6d624cf698b6cd91b0e68335 (127.9.123.67:42333)
I20260926 15:02:38.590320 11627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf54b2c6363d49f9b0b93fc33fcc5208" candidate_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "649129489c4049588da477db3e4fba75"
I20260926 15:02:38.590324 11907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf54b2c6363d49f9b0b93fc33fcc5208" candidate_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a43d1b2c6d624cf698b6cd91b0e68335"
I20260926 15:02:38.590517 11627 raft_consensus.cc:3037] T cf54b2c6363d49f9b0b93fc33fcc5208 P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:38.590564 11907 raft_consensus.cc:3037] T cf54b2c6363d49f9b0b93fc33fcc5208 P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:38.592705 11627 raft_consensus.cc:2454] T cf54b2c6363d49f9b0b93fc33fcc5208 P 649129489c4049588da477db3e4fba75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc5a333bcb14ce39f96fe2fef5c9f7a in term 1.
I20260926 15:02:38.592752 11907 raft_consensus.cc:2454] T cf54b2c6363d49f9b0b93fc33fcc5208 P a43d1b2c6d624cf698b6cd91b0e68335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc5a333bcb14ce39f96fe2fef5c9f7a in term 1.
I20260926 15:02:38.593194 11698 leader_election.cc:302] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc5a333bcb14ce39f96fe2fef5c9f7a, a43d1b2c6d624cf698b6cd91b0e68335; no voters:
I20260926 15:02:38.594149 12002 raft_consensus.cc:2781] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:38.594606 12002 raft_consensus.cc:686] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 1 LEADER]: Becoming Leader. State: Replica: 1fc5a333bcb14ce39f96fe2fef5c9f7a, State: Running, Role: LEADER
I20260926 15:02:38.595126 12002 consensus_queue.cc:237] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:38.597834 12007 tablet_replica.cc:439] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: TxnStatusTablet state changed. Reason: New leader 1fc5a333bcb14ce39f96fe2fef5c9f7a. Latest consensus state: current_term: 1 leader_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } } }
I20260926 15:02:38.598312 12007 tablet_replica.cc:442] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: This TxnStatusTablet replica's current role is: LEADER
I20260926 15:02:38.599040 12009 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 15:02:38.599303 12009 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 15:02:38.599732 11468 catalog_manager.cc:5495] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc5a333bcb14ce39f96fe2fef5c9f7a (127.9.123.66). New cstate: current_term: 1 leader_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } health_report { overall_health: UNKNOWN } } }
I20260926 15:02:38.623766 11995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:38.616756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:45177 (local address 127.0.0.1:47612)
0926 15:02:38.616989 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:38.617008 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:38.617173 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:38.617829 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:38.617832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:38.617843 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 15:02:38.617996 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:38.618003 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.619518 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.619523 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:38.620865 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:38.620881 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.621945 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.621949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:38.622101 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:38.622169 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 15:02:38.623281 (+ 1112us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 15:02:38.623301 (+ 20us) client_negotiation.cc:769] Sending connection context
0926 15:02:38.623374 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 15:02:38.623533 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260926 15:02:38.623842 11971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:38.616757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47612 (local address 127.9.123.66:45177)
0926 15:02:38.616943 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:38.616950 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:38.617081 (+ 131us) server_negotiation.cc:407] Connection header received
0926 15:02:38.617386 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:38.617392 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:38.617519 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:38.617779 (+ 260us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:02:38.618128 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.619346 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.621028 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.621771 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.621905 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:38.622388 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:02:38.623164 (+ 776us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:02:38.623310 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:38.623413 (+ 103us) server_negotiation.cc:299] Negotiation successful
0926 15:02:38.623507 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 15:02:38.632264 11907 raft_consensus.cc:1261] T cf54b2c6363d49f9b0b93fc33fcc5208 P a43d1b2c6d624cf698b6cd91b0e68335 [term 1 FOLLOWER]: Refusing update from remote peer 1fc5a333bcb14ce39f96fe2fef5c9f7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:02:38.632242 11627 raft_consensus.cc:1261] T cf54b2c6363d49f9b0b93fc33fcc5208 P 649129489c4049588da477db3e4fba75 [term 1 FOLLOWER]: Refusing update from remote peer 1fc5a333bcb14ce39f96fe2fef5c9f7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:02:38.633291 12007 consensus_queue.cc:1035] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:38.633690 12002 consensus_queue.cc:1035] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:38.649154 12003 tablet_replica.cc:439] T cf54b2c6363d49f9b0b93fc33fcc5208 P a43d1b2c6d624cf698b6cd91b0e68335: TxnStatusTablet state changed. Reason: New leader 1fc5a333bcb14ce39f96fe2fef5c9f7a. Latest consensus state: current_term: 1 leader_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } } }
I20260926 15:02:38.649509 12003 tablet_replica.cc:442] T cf54b2c6363d49f9b0b93fc33fcc5208 P a43d1b2c6d624cf698b6cd91b0e68335: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 15:02:38.651010 11983 tablet_replica.cc:439] T cf54b2c6363d49f9b0b93fc33fcc5208 P 649129489c4049588da477db3e4fba75: TxnStatusTablet state changed. Reason: New leader 1fc5a333bcb14ce39f96fe2fef5c9f7a. Latest consensus state: current_term: 1 leader_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } } }
I20260926 15:02:38.651288 11983 tablet_replica.cc:442] T cf54b2c6363d49f9b0b93fc33fcc5208 P 649129489c4049588da477db3e4fba75: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 15:02:38.652490 12012 mvcc.cc:205] Tried to move back new op lower bound from 7333621590547648512 to 7333621590409293824. Current Snapshot: MvccSnapshot[applied={T|T < 7333621590547648512}]
I20260926 15:02:38.659574 12013 mvcc.cc:205] Tried to move back new op lower bound from 7333621590547648512 to 7333621590409293824. Current Snapshot: MvccSnapshot[applied={T|T < 7333621590547648512}]
I20260926 15:02:38.662225 12002 tablet_replica.cc:439] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } } }
I20260926 15:02:38.662516 12002 tablet_replica.cc:442] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: This TxnStatusTablet replica's current role is: LEADER
I20260926 15:02:38.665288 12002 tablet_replica.cc:439] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } } }
I20260926 15:02:38.665565 12002 tablet_replica.cc:442] T cf54b2c6363d49f9b0b93fc33fcc5208 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: This TxnStatusTablet replica's current role is: LEADER
I20260926 15:02:38.671173 11983 tablet_replica.cc:439] T cf54b2c6363d49f9b0b93fc33fcc5208 P 649129489c4049588da477db3e4fba75: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } } }
I20260926 15:02:38.671409 11983 tablet_replica.cc:442] T cf54b2c6363d49f9b0b93fc33fcc5208 P 649129489c4049588da477db3e4fba75: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 15:02:38.682960 12003 tablet_replica.cc:439] T cf54b2c6363d49f9b0b93fc33fcc5208 P a43d1b2c6d624cf698b6cd91b0e68335: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } } }
I20260926 15:02:38.683233 12003 tablet_replica.cc:442] T cf54b2c6363d49f9b0b93fc33fcc5208 P a43d1b2c6d624cf698b6cd91b0e68335: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 15:02:38.690124 12005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:38.678249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50720 (local address 127.9.123.65:34481)
0926 15:02:38.678445 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:38.678451 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:38.681657 (+ 3206us) server_negotiation.cc:407] Connection header received
0926 15:02:38.681886 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:38.681892 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:38.682002 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:38.682147 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:02:38.682840 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.683932 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.687272 (+ 3340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.687938 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.688062 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:38.688589 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:02:38.689321 (+ 732us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:02:38.689463 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:38.689560 (+ 97us) server_negotiation.cc:299] Negotiation successful
0926 15:02:38.689650 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 15:02:38.708302 11971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:38.701478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:51821 (local address 127.9.123.66:45177)
0926 15:02:38.701654 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:38.701658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:38.702195 (+ 537us) server_negotiation.cc:407] Connection header received
0926 15:02:38.702359 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:38.702365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:38.702487 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:38.702589 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:02:38.703014 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.704180 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.705617 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.706362 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.706508 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:38.707400 (+ 892us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:02:38.707521 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:02:38.707616 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:38.707861 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 15:02:38.707942 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 15:02:38.708324 12024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:38.701496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:45177 (local address 127.9.123.65:51821)
0926 15:02:38.702087 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:38.702103 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:38.702229 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:38.702615 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:38.702618 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:38.702628 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 15:02:38.702868 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:38.702876 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.704324 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.704329 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:38.705462 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:38.705475 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.706839 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.706846 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:38.707044 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:38.707128 (+ 84us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 15:02:38.707639 (+ 511us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 15:02:38.707658 (+ 19us) client_negotiation.cc:769] Sending connection context
0926 15:02:38.707751 (+ 93us) client_negotiation.cc:240] Negotiation successful
0926 15:02:38.707918 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":296,"threads_started":1}
I20260926 15:02:38.719174 11907 raft_consensus.cc:1261] T 5b9cd727ab3a481e9c3a386908f33f49 P a43d1b2c6d624cf698b6cd91b0e68335 [term 1 FOLLOWER]: Refusing update from remote peer 649129489c4049588da477db3e4fba75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:02:38.719496 11767 raft_consensus.cc:1261] T 5b9cd727ab3a481e9c3a386908f33f49 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 1 FOLLOWER]: Refusing update from remote peer 649129489c4049588da477db3e4fba75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:02:38.719897 11987 consensus_queue.cc:1035] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:38.720753 11983 consensus_queue.cc:1035] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:38.730182 12014 mvcc.cc:205] Tried to move back new op lower bound from 7333621590906732544 to 7333621589143244800. Current Snapshot: MvccSnapshot[applied={T|T < 7333621590906732544}]
I20260926 15:02:38.760555 12005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:38.754556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:58251 (local address 127.9.123.65:34481)
0926 15:02:38.754710 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:38.754713 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:38.755439 (+ 726us) server_negotiation.cc:407] Connection header received
0926 15:02:38.755620 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:38.755626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:38.755729 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:38.755824 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:02:38.756384 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.757162 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.758763 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.759185 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.759292 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:38.759825 (+ 533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:02:38.759973 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:02:38.760055 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:38.760157 (+ 102us) server_negotiation.cc:299] Negotiation successful
0926 15:02:38.760234 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 15:02:38.760885 12029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:38.754567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:34481 (local address 127.9.123.66:58251)
0926 15:02:38.755335 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:38.755353 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:38.755488 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:38.755842 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:38.755847 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:38.755859 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 15:02:38.756227 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:38.756237 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.757282 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.757288 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:38.758619 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:38.758631 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:38.759369 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:38.759375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:38.759563 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:38.759645 (+ 82us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 15:02:38.760056 (+ 411us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 15:02:38.760068 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 15:02:38.760139 (+ 71us) client_negotiation.cc:240] Negotiation successful
0926 15:02:38.760283 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":429,"threads_started":1}
I20260926 15:02:38.806794 11469 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:02:38.807070 11469 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:02:38.810456 11469 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c: Sending DeleteTablet for 3 replicas of tablet 5b9cd727ab3a481e9c3a386908f33f49
I20260926 15:02:38.811837 11887 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b9cd727ab3a481e9c3a386908f33f49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:02:38 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:39884
I20260926 15:02:38.811932 11607 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b9cd727ab3a481e9c3a386908f33f49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:02:38 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:50718
I20260926 15:02:38.812770 11747 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b9cd727ab3a481e9c3a386908f33f49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:02:38 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:47604
I20260926 15:02:38.814173 11469 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:02:38.814888 11469 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:02:38.824796 11886 tablet_service.cc:1467] Processing CreateTablet for tablet 6c0059358c9940c0be3180a3f24155ee (DEFAULT_TABLE table=test-table [id=10b11dcd8e9243e09d6942c04cf64c41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:38.825274 11606 tablet_service.cc:1467] Processing CreateTablet for tablet 6c0059358c9940c0be3180a3f24155ee (DEFAULT_TABLE table=test-table [id=10b11dcd8e9243e09d6942c04cf64c41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:38.825459 11886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c0059358c9940c0be3180a3f24155ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:38.825807 11606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c0059358c9940c0be3180a3f24155ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:38.829684 11746 tablet_service.cc:1467] Processing CreateTablet for tablet 6c0059358c9940c0be3180a3f24155ee (DEFAULT_TABLE table=test-table [id=10b11dcd8e9243e09d6942c04cf64c41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:38.830392 11746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c0059358c9940c0be3180a3f24155ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:38.832368 11997 tablet_bootstrap.cc:492] T 6c0059358c9940c0be3180a3f24155ee P a43d1b2c6d624cf698b6cd91b0e68335: Bootstrap starting.
I20260926 15:02:38.834072 11997 tablet_bootstrap.cc:654] T 6c0059358c9940c0be3180a3f24155ee P a43d1b2c6d624cf698b6cd91b0e68335: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:38.842295 11996 tablet_bootstrap.cc:492] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75: Bootstrap starting.
I20260926 15:02:38.842908 11998 tablet_bootstrap.cc:492] T 6c0059358c9940c0be3180a3f24155ee P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Bootstrap starting.
I20260926 15:02:38.844305 11996 tablet_bootstrap.cc:654] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:38.844623 11998 tablet_bootstrap.cc:654] T 6c0059358c9940c0be3180a3f24155ee P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:38.857973 11998 tablet_bootstrap.cc:492] T 6c0059358c9940c0be3180a3f24155ee P 1fc5a333bcb14ce39f96fe2fef5c9f7a: No bootstrap required, opened a new log
I20260926 15:02:38.858279 11998 ts_tablet_manager.cc:1397] T 6c0059358c9940c0be3180a3f24155ee P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.009s
I20260926 15:02:38.858780 12034 tablet_replica.cc:330] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75: stopping tablet replica
I20260926 15:02:38.859301 12034 raft_consensus.cc:2229] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75 [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:02:38.859285 11998 raft_consensus.cc:348] T 6c0059358c9940c0be3180a3f24155ee P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:38.859777 11998 raft_consensus.cc:374] T 6c0059358c9940c0be3180a3f24155ee P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:38.859912 11998 raft_consensus.cc:729] T 6c0059358c9940c0be3180a3f24155ee P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc5a333bcb14ce39f96fe2fef5c9f7a, State: Initialized, Role: FOLLOWER
I20260926 15:02:38.860005 12034 raft_consensus.cc:2258] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:38.860814 11998 consensus_queue.cc:260] T 6c0059358c9940c0be3180a3f24155ee P 1fc5a333bcb14ce39f96fe2fef5c9f7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:38.861481 11996 tablet_bootstrap.cc:492] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75: No bootstrap required, opened a new log
I20260926 15:02:38.861671 11996 ts_tablet_manager.cc:1397] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.006s
I20260926 15:02:38.862785 11998 ts_tablet_manager.cc:1428] T 6c0059358c9940c0be3180a3f24155ee P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260926 15:02:38.863449 12035 tablet_replica.cc:330] T 5b9cd727ab3a481e9c3a386908f33f49 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: stopping tablet replica
I20260926 15:02:38.863942 12035 raft_consensus.cc:2229] T 5b9cd727ab3a481e9c3a386908f33f49 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:38.864368 12035 raft_consensus.cc:2258] T 5b9cd727ab3a481e9c3a386908f33f49 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:38.862450 11996 raft_consensus.cc:348] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:38.865483 11996 raft_consensus.cc:374] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:38.865571 11996 raft_consensus.cc:729] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 649129489c4049588da477db3e4fba75, State: Initialized, Role: FOLLOWER
I20260926 15:02:38.865833 11996 consensus_queue.cc:260] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:38.866354 11996 ts_tablet_manager.cc:1428] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260926 15:02:38.867676 12035 ts_tablet_manager.cc:1905] T 5b9cd727ab3a481e9c3a386908f33f49 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:38.868242 12033 tablet_replica.cc:330] T 5b9cd727ab3a481e9c3a386908f33f49 P a43d1b2c6d624cf698b6cd91b0e68335: stopping tablet replica
I20260926 15:02:38.868857 11997 tablet_bootstrap.cc:492] T 6c0059358c9940c0be3180a3f24155ee P a43d1b2c6d624cf698b6cd91b0e68335: No bootstrap required, opened a new log
I20260926 15:02:38.869144 11997 ts_tablet_manager.cc:1397] T 6c0059358c9940c0be3180a3f24155ee P a43d1b2c6d624cf698b6cd91b0e68335: Time spent bootstrapping tablet: real 0.037s user 0.011s sys 0.003s
I20260926 15:02:38.869551 12033 raft_consensus.cc:2229] T 5b9cd727ab3a481e9c3a386908f33f49 P a43d1b2c6d624cf698b6cd91b0e68335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:38.869961 12033 raft_consensus.cc:2258] T 5b9cd727ab3a481e9c3a386908f33f49 P a43d1b2c6d624cf698b6cd91b0e68335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:38.871923 12033 ts_tablet_manager.cc:1905] T 5b9cd727ab3a481e9c3a386908f33f49 P a43d1b2c6d624cf698b6cd91b0e68335: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:38.872504 11997 raft_consensus.cc:348] T 6c0059358c9940c0be3180a3f24155ee P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:38.872841 11997 raft_consensus.cc:374] T 6c0059358c9940c0be3180a3f24155ee P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:38.872967 11997 raft_consensus.cc:729] T 6c0059358c9940c0be3180a3f24155ee P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a43d1b2c6d624cf698b6cd91b0e68335, State: Initialized, Role: FOLLOWER
I20260926 15:02:38.873313 11997 consensus_queue.cc:260] T 6c0059358c9940c0be3180a3f24155ee P a43d1b2c6d624cf698b6cd91b0e68335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:38.873921 11997 ts_tablet_manager.cc:1428] T 6c0059358c9940c0be3180a3f24155ee P a43d1b2c6d624cf698b6cd91b0e68335: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20260926 15:02:38.881762 12035 ts_tablet_manager.cc:1918] T 5b9cd727ab3a481e9c3a386908f33f49 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 15:02:38.881929 12035 log.cc:1196] T 5b9cd727ab3a481e9c3a386908f33f49 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/5b9cd727ab3a481e9c3a386908f33f49
I20260926 15:02:38.882464 12035 ts_tablet_manager.cc:1939] T 5b9cd727ab3a481e9c3a386908f33f49 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Deleting consensus metadata
I20260926 15:02:38.882889 12034 ts_tablet_manager.cc:1905] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:38.884328 11456 catalog_manager.cc:4841] TS 1fc5a333bcb14ce39f96fe2fef5c9f7a (127.9.123.66:45177): tablet 5b9cd727ab3a481e9c3a386908f33f49 (table test-table [id=0e4041b7e6a241068e6d0fda3b465c50]) successfully deleted
I20260926 15:02:38.891749 12033 ts_tablet_manager.cc:1918] T 5b9cd727ab3a481e9c3a386908f33f49 P a43d1b2c6d624cf698b6cd91b0e68335: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 15:02:38.891973 12033 log.cc:1196] T 5b9cd727ab3a481e9c3a386908f33f49 P a43d1b2c6d624cf698b6cd91b0e68335: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/5b9cd727ab3a481e9c3a386908f33f49
I20260926 15:02:38.892372 12034 ts_tablet_manager.cc:1918] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 15:02:38.892477 12033 ts_tablet_manager.cc:1939] T 5b9cd727ab3a481e9c3a386908f33f49 P a43d1b2c6d624cf698b6cd91b0e68335: Deleting consensus metadata
I20260926 15:02:38.892707 12034 log.cc:1196] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-0/wal/wals/5b9cd727ab3a481e9c3a386908f33f49
I20260926 15:02:38.893339 12034 ts_tablet_manager.cc:1939] T 5b9cd727ab3a481e9c3a386908f33f49 P 649129489c4049588da477db3e4fba75: Deleting consensus metadata
I20260926 15:02:38.894474 11455 catalog_manager.cc:4841] TS a43d1b2c6d624cf698b6cd91b0e68335 (127.9.123.67:42333): tablet 5b9cd727ab3a481e9c3a386908f33f49 (table test-table [id=0e4041b7e6a241068e6d0fda3b465c50]) successfully deleted
I20260926 15:02:38.895216 11456 catalog_manager.cc:4841] TS 649129489c4049588da477db3e4fba75 (127.9.123.65:34481): tablet 5b9cd727ab3a481e9c3a386908f33f49 (table test-table [id=0e4041b7e6a241068e6d0fda3b465c50]) successfully deleted
I20260926 15:02:39.009640 12027 raft_consensus.cc:482] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:39.009881 12027 raft_consensus.cc:504] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:39.010841 12027 leader_election.cc:288] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fc5a333bcb14ce39f96fe2fef5c9f7a (127.9.123.66:45177), a43d1b2c6d624cf698b6cd91b0e68335 (127.9.123.67:42333)
I20260926 15:02:39.011637 11907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c0059358c9940c0be3180a3f24155ee" candidate_uuid: "649129489c4049588da477db3e4fba75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" is_pre_election: true
I20260926 15:02:39.011618 11767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c0059358c9940c0be3180a3f24155ee" candidate_uuid: "649129489c4049588da477db3e4fba75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" is_pre_election: true
I20260926 15:02:39.011880 11907 raft_consensus.cc:2454] T 6c0059358c9940c0be3180a3f24155ee P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 649129489c4049588da477db3e4fba75 in term 0.
I20260926 15:02:39.011916 11767 raft_consensus.cc:2454] T 6c0059358c9940c0be3180a3f24155ee P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 649129489c4049588da477db3e4fba75 in term 0.
I20260926 15:02:39.012354 11558 leader_election.cc:302] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 649129489c4049588da477db3e4fba75, a43d1b2c6d624cf698b6cd91b0e68335; no voters:
I20260926 15:02:39.012737 12027 raft_consensus.cc:2781] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:39.012871 12027 raft_consensus.cc:482] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:39.012964 12027 raft_consensus.cc:3037] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:39.016533 12027 raft_consensus.cc:504] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:39.017354 12027 leader_election.cc:288] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75 [CANDIDATE]: Term 1 election: Requested vote from peers 1fc5a333bcb14ce39f96fe2fef5c9f7a (127.9.123.66:45177), a43d1b2c6d624cf698b6cd91b0e68335 (127.9.123.67:42333)
I20260926 15:02:39.017915 11767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c0059358c9940c0be3180a3f24155ee" candidate_uuid: "649129489c4049588da477db3e4fba75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a"
I20260926 15:02:39.018154 11767 raft_consensus.cc:3037] T 6c0059358c9940c0be3180a3f24155ee P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:39.018481 11907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c0059358c9940c0be3180a3f24155ee" candidate_uuid: "649129489c4049588da477db3e4fba75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a43d1b2c6d624cf698b6cd91b0e68335"
I20260926 15:02:39.018695 11907 raft_consensus.cc:3037] T 6c0059358c9940c0be3180a3f24155ee P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:39.021415 11767 raft_consensus.cc:2454] T 6c0059358c9940c0be3180a3f24155ee P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 649129489c4049588da477db3e4fba75 in term 1.
I20260926 15:02:39.021574 11907 raft_consensus.cc:2454] T 6c0059358c9940c0be3180a3f24155ee P a43d1b2c6d624cf698b6cd91b0e68335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 649129489c4049588da477db3e4fba75 in term 1.
I20260926 15:02:39.021946 11559 leader_election.cc:302] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc5a333bcb14ce39f96fe2fef5c9f7a, 649129489c4049588da477db3e4fba75; no voters:
I20260926 15:02:39.022354 12027 raft_consensus.cc:2781] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:39.022598 12027 raft_consensus.cc:686] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75 [term 1 LEADER]: Becoming Leader. State: Replica: 649129489c4049588da477db3e4fba75, State: Running, Role: LEADER
I20260926 15:02:39.022873 12027 consensus_queue.cc:237] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:39.025733 11470 catalog_manager.cc:5495] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 649129489c4049588da477db3e4fba75 (127.9.123.65). New cstate: current_term: 1 leader_uuid: "649129489c4049588da477db3e4fba75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } health_report { overall_health: UNKNOWN } } }
I20260926 15:02:39.110829 11906 raft_consensus.cc:1261] T 6c0059358c9940c0be3180a3f24155ee P a43d1b2c6d624cf698b6cd91b0e68335 [term 1 FOLLOWER]: Refusing update from remote peer 649129489c4049588da477db3e4fba75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:02:39.112073 11983 consensus_queue.cc:1035] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:39.114018 11767 raft_consensus.cc:1261] T 6c0059358c9940c0be3180a3f24155ee P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 1 FOLLOWER]: Refusing update from remote peer 649129489c4049588da477db3e4fba75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:02:39.115062 12027 consensus_queue.cc:1035] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 15:02:39.138798 12013 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 15:02:40.725979 11470 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:02:40.726234 11470 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:02:40.730918 11470 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c: Sending DeleteTablet for 3 replicas of tablet 6c0059358c9940c0be3180a3f24155ee
I20260926 15:02:40.731863 11887 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c0059358c9940c0be3180a3f24155ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:02:40 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:39884
I20260926 15:02:40.731841 11606 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c0059358c9940c0be3180a3f24155ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:02:40 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:50718
I20260926 15:02:40.731899 11747 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c0059358c9940c0be3180a3f24155ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:02:40 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:47604
I20260926 15:02:40.732554 12065 tablet_replica.cc:330] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75: stopping tablet replica
I20260926 15:02:40.732847 12065 raft_consensus.cc:2229] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75 [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:02:40.733323 12065 raft_consensus.cc:2258] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:40.734568 12065 ts_tablet_manager.cc:1905] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:40.742789 12065 ts_tablet_manager.cc:1918] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 15:02:40.743070 12065 log.cc:1196] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-0/wal/wals/6c0059358c9940c0be3180a3f24155ee
I20260926 15:02:40.743575 12065 ts_tablet_manager.cc:1939] T 6c0059358c9940c0be3180a3f24155ee P 649129489c4049588da477db3e4fba75: Deleting consensus metadata
I20260926 15:02:40.744994 11456 catalog_manager.cc:4841] TS 649129489c4049588da477db3e4fba75 (127.9.123.65:34481): tablet 6c0059358c9940c0be3180a3f24155ee (table test-table [id=10b11dcd8e9243e09d6942c04cf64c41]) successfully deleted
I20260926 15:02:40.746279 12064 tablet_replica.cc:330] T 6c0059358c9940c0be3180a3f24155ee P 1fc5a333bcb14ce39f96fe2fef5c9f7a: stopping tablet replica
I20260926 15:02:40.746502 12064 raft_consensus.cc:2229] T 6c0059358c9940c0be3180a3f24155ee P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:40.746740 12064 raft_consensus.cc:2258] T 6c0059358c9940c0be3180a3f24155ee P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:40.747341 12063 tablet_replica.cc:330] T 6c0059358c9940c0be3180a3f24155ee P a43d1b2c6d624cf698b6cd91b0e68335: stopping tablet replica
I20260926 15:02:40.747541 12063 raft_consensus.cc:2229] T 6c0059358c9940c0be3180a3f24155ee P a43d1b2c6d624cf698b6cd91b0e68335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:40.747757 12063 raft_consensus.cc:2258] T 6c0059358c9940c0be3180a3f24155ee P a43d1b2c6d624cf698b6cd91b0e68335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:40.748238 12064 ts_tablet_manager.cc:1905] T 6c0059358c9940c0be3180a3f24155ee P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:40.748589 12063 ts_tablet_manager.cc:1905] T 6c0059358c9940c0be3180a3f24155ee P a43d1b2c6d624cf698b6cd91b0e68335: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:40.752874 12064 ts_tablet_manager.cc:1918] T 6c0059358c9940c0be3180a3f24155ee P 1fc5a333bcb14ce39f96fe2fef5c9f7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 15:02:40.753029 12064 log.cc:1196] T 6c0059358c9940c0be3180a3f24155ee P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/6c0059358c9940c0be3180a3f24155ee
I20260926 15:02:40.753535 12064 ts_tablet_manager.cc:1939] T 6c0059358c9940c0be3180a3f24155ee P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Deleting consensus metadata
I20260926 15:02:40.754629 12063 ts_tablet_manager.cc:1918] T 6c0059358c9940c0be3180a3f24155ee P a43d1b2c6d624cf698b6cd91b0e68335: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 15:02:40.754644 11456 catalog_manager.cc:4841] TS 1fc5a333bcb14ce39f96fe2fef5c9f7a (127.9.123.66:45177): tablet 6c0059358c9940c0be3180a3f24155ee (table test-table [id=10b11dcd8e9243e09d6942c04cf64c41]) successfully deleted
I20260926 15:02:40.754757 12063 log.cc:1196] T 6c0059358c9940c0be3180a3f24155ee P a43d1b2c6d624cf698b6cd91b0e68335: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/6c0059358c9940c0be3180a3f24155ee
I20260926 15:02:40.755167 12063 ts_tablet_manager.cc:1939] T 6c0059358c9940c0be3180a3f24155ee P a43d1b2c6d624cf698b6cd91b0e68335: Deleting consensus metadata
I20260926 15:02:40.756111 11455 catalog_manager.cc:4841] TS a43d1b2c6d624cf698b6cd91b0e68335 (127.9.123.67:42333): tablet 6c0059358c9940c0be3180a3f24155ee (table test-table [id=10b11dcd8e9243e09d6942c04cf64c41]) successfully deleted
Sep 26 15:02:40 dist-test-slave-7153 krb5kdc[11425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434960, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 15:02:40 dist-test-slave-7153 krb5kdc[11425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434960, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:40.795969 12073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:40.773859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48988 (local address 127.9.123.126:38981)
0926 15:02:40.774483 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:40.774487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:40.774509 (+ 22us) server_negotiation.cc:407] Connection header received
0926 15:02:40.774658 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:40.774665 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:40.774732 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:40.774831 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:40.776093 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:40.776877 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:40.778261 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:40.778941 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:40.779110 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:40.788971 (+ 9861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:40.788980 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:40.788986 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:40.789035 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:40.792894 (+ 3859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:40.793854 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:40.793860 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:40.793864 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:40.794045 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:40.794541 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:40.794545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:40.794548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:40.794896 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:40.795088 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:40.795433 (+ 345us) server_negotiation.cc:299] Negotiation successful
0926 15:02:40.795615 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":387,"threads_started":1}
I20260926 15:02:40.799806 11470 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48988:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:02:40.800408 11470 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:02:40.806931 11747 tablet_service.cc:1467] Processing CreateTablet for tablet b589127c2a45441bb78f306389d5ccf2 (DEFAULT_TABLE table=test-table [id=acdc4e0266a0418fb850b37151e8f869]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:40.806891 11606 tablet_service.cc:1467] Processing CreateTablet for tablet b589127c2a45441bb78f306389d5ccf2 (DEFAULT_TABLE table=test-table [id=acdc4e0266a0418fb850b37151e8f869]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:40.807158 11887 tablet_service.cc:1467] Processing CreateTablet for tablet b589127c2a45441bb78f306389d5ccf2 (DEFAULT_TABLE table=test-table [id=acdc4e0266a0418fb850b37151e8f869]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:40.807328 11747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b589127c2a45441bb78f306389d5ccf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:40.807346 11606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b589127c2a45441bb78f306389d5ccf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:40.807546 11887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b589127c2a45441bb78f306389d5ccf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:40.814620 12076 tablet_bootstrap.cc:492] T b589127c2a45441bb78f306389d5ccf2 P 649129489c4049588da477db3e4fba75: Bootstrap starting.
I20260926 15:02:40.815368 12074 tablet_bootstrap.cc:492] T b589127c2a45441bb78f306389d5ccf2 P a43d1b2c6d624cf698b6cd91b0e68335: Bootstrap starting.
I20260926 15:02:40.816445 12076 tablet_bootstrap.cc:654] T b589127c2a45441bb78f306389d5ccf2 P 649129489c4049588da477db3e4fba75: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:40.817224 12074 tablet_bootstrap.cc:654] T b589127c2a45441bb78f306389d5ccf2 P a43d1b2c6d624cf698b6cd91b0e68335: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:40.819104 12076 tablet_bootstrap.cc:492] T b589127c2a45441bb78f306389d5ccf2 P 649129489c4049588da477db3e4fba75: No bootstrap required, opened a new log
I20260926 15:02:40.819226 12076 ts_tablet_manager.cc:1397] T b589127c2a45441bb78f306389d5ccf2 P 649129489c4049588da477db3e4fba75: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 15:02:40.818573 12075 tablet_bootstrap.cc:492] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Bootstrap starting.
I20260926 15:02:40.819856 12076 raft_consensus.cc:348] T b589127c2a45441bb78f306389d5ccf2 P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:40.820013 12074 tablet_bootstrap.cc:492] T b589127c2a45441bb78f306389d5ccf2 P a43d1b2c6d624cf698b6cd91b0e68335: No bootstrap required, opened a new log
I20260926 15:02:40.820079 12076 raft_consensus.cc:374] T b589127c2a45441bb78f306389d5ccf2 P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:40.820140 12074 ts_tablet_manager.cc:1397] T b589127c2a45441bb78f306389d5ccf2 P a43d1b2c6d624cf698b6cd91b0e68335: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260926 15:02:40.820199 12076 raft_consensus.cc:729] T b589127c2a45441bb78f306389d5ccf2 P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 649129489c4049588da477db3e4fba75, State: Initialized, Role: FOLLOWER
I20260926 15:02:40.820441 12076 consensus_queue.cc:260] T b589127c2a45441bb78f306389d5ccf2 P 649129489c4049588da477db3e4fba75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:40.820927 12076 ts_tablet_manager.cc:1428] T b589127c2a45441bb78f306389d5ccf2 P 649129489c4049588da477db3e4fba75: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 15:02:40.821074 12074 raft_consensus.cc:348] T b589127c2a45441bb78f306389d5ccf2 P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:40.821538 12074 raft_consensus.cc:374] T b589127c2a45441bb78f306389d5ccf2 P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:40.821589 12075 tablet_bootstrap.cc:654] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:40.821645 12074 raft_consensus.cc:729] T b589127c2a45441bb78f306389d5ccf2 P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a43d1b2c6d624cf698b6cd91b0e68335, State: Initialized, Role: FOLLOWER
I20260926 15:02:40.821898 12074 consensus_queue.cc:260] T b589127c2a45441bb78f306389d5ccf2 P a43d1b2c6d624cf698b6cd91b0e68335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:40.822745 12074 ts_tablet_manager.cc:1428] T b589127c2a45441bb78f306389d5ccf2 P a43d1b2c6d624cf698b6cd91b0e68335: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 15:02:40.824074 12075 tablet_bootstrap.cc:492] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: No bootstrap required, opened a new log
I20260926 15:02:40.824208 12075 ts_tablet_manager.cc:1397] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260926 15:02:40.824970 12075 raft_consensus.cc:348] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:40.825227 12075 raft_consensus.cc:374] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:40.825320 12075 raft_consensus.cc:729] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc5a333bcb14ce39f96fe2fef5c9f7a, State: Initialized, Role: FOLLOWER
I20260926 15:02:40.825527 12075 consensus_queue.cc:260] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:40.825999 12075 ts_tablet_manager.cc:1428] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 15:02:40.906266 12044 raft_consensus.cc:482] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:40.906487 12044 raft_consensus.cc:504] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:40.907256 12044 leader_election.cc:288] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 649129489c4049588da477db3e4fba75 (127.9.123.65:34481), a43d1b2c6d624cf698b6cd91b0e68335 (127.9.123.67:42333)
I20260926 15:02:40.907781 11627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b589127c2a45441bb78f306389d5ccf2" candidate_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "649129489c4049588da477db3e4fba75" is_pre_election: true
I20260926 15:02:40.907806 11906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b589127c2a45441bb78f306389d5ccf2" candidate_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" is_pre_election: true
I20260926 15:02:40.907986 11627 raft_consensus.cc:2454] T b589127c2a45441bb78f306389d5ccf2 P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc5a333bcb14ce39f96fe2fef5c9f7a in term 0.
I20260926 15:02:40.908035 11906 raft_consensus.cc:2454] T b589127c2a45441bb78f306389d5ccf2 P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fc5a333bcb14ce39f96fe2fef5c9f7a in term 0.
I20260926 15:02:40.908445 11699 leader_election.cc:302] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc5a333bcb14ce39f96fe2fef5c9f7a, 649129489c4049588da477db3e4fba75; no voters:
I20260926 15:02:40.908708 12044 raft_consensus.cc:2781] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:40.908815 12044 raft_consensus.cc:482] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:40.908878 12044 raft_consensus.cc:3037] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:40.910943 12044 raft_consensus.cc:504] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:40.911437 12044 leader_election.cc:288] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [CANDIDATE]: Term 1 election: Requested vote from peers 649129489c4049588da477db3e4fba75 (127.9.123.65:34481), a43d1b2c6d624cf698b6cd91b0e68335 (127.9.123.67:42333)
I20260926 15:02:40.911906 11906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b589127c2a45441bb78f306389d5ccf2" candidate_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a43d1b2c6d624cf698b6cd91b0e68335"
I20260926 15:02:40.911931 11627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b589127c2a45441bb78f306389d5ccf2" candidate_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "649129489c4049588da477db3e4fba75"
I20260926 15:02:40.912137 11906 raft_consensus.cc:3037] T b589127c2a45441bb78f306389d5ccf2 P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:40.912133 11627 raft_consensus.cc:3037] T b589127c2a45441bb78f306389d5ccf2 P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:40.914951 11627 raft_consensus.cc:2454] T b589127c2a45441bb78f306389d5ccf2 P 649129489c4049588da477db3e4fba75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc5a333bcb14ce39f96fe2fef5c9f7a in term 1.
I20260926 15:02:40.915359 11906 raft_consensus.cc:2454] T b589127c2a45441bb78f306389d5ccf2 P a43d1b2c6d624cf698b6cd91b0e68335 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fc5a333bcb14ce39f96fe2fef5c9f7a in term 1.
I20260926 15:02:40.915400 11699 leader_election.cc:302] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fc5a333bcb14ce39f96fe2fef5c9f7a, 649129489c4049588da477db3e4fba75; no voters:
I20260926 15:02:40.915696 12044 raft_consensus.cc:2781] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:40.915856 12044 raft_consensus.cc:686] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 1 LEADER]: Becoming Leader. State: Replica: 1fc5a333bcb14ce39f96fe2fef5c9f7a, State: Running, Role: LEADER
I20260926 15:02:40.916126 12044 consensus_queue.cc:237] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:40.918509 11470 catalog_manager.cc:5495] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fc5a333bcb14ce39f96fe2fef5c9f7a (127.9.123.66). New cstate: current_term: 1 leader_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } health_report { overall_health: UNKNOWN } } }
I20260926 15:02:40.958482 12084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:40.947075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35868 (local address 127.9.123.66:45177)
0926 15:02:40.953699 (+ 6624us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:40.953705 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:40.953732 (+ 27us) server_negotiation.cc:407] Connection header received
0926 15:02:40.953800 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:40.953804 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:40.953877 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:40.954002 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:02:40.954518 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:40.955276 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:40.956650 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:40.957077 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:40.957223 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:40.957677 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:02:40.957799 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:02:40.957879 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:40.958067 (+ 188us) server_negotiation.cc:299] Negotiation successful
0926 15:02:40.958134 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6468,"thread_start_us":6295,"threads_started":1}
I20260926 15:02:40.972430 12084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:40.967458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:40113 (local address 127.9.123.66:45177)
0926 15:02:40.967598 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:40.967602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:40.968061 (+ 459us) server_negotiation.cc:407] Connection header received
0926 15:02:40.968180 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:40.968183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:40.968247 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:40.968322 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:02:40.968799 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:40.969636 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:40.970986 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:40.971356 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:40.971495 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:40.971833 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:02:40.971931 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:02:40.972031 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:40.972126 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 15:02:40.972185 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 15:02:40.972513 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:40.967379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:45177 (local address 127.9.123.66:40113)
0926 15:02:40.967936 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:40.967956 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:40.968081 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:40.968395 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:40.968398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:40.968409 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 15:02:40.968633 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:40.968640 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:40.969783 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:40.969787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:40.970810 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:40.970821 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:40.971537 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:40.971542 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:40.971668 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:40.971706 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 15:02:40.972036 (+ 330us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 15:02:40.972044 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 15:02:40.972103 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 15:02:40.972205 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":283,"threads_started":1}
I20260926 15:02:40.980513 11627 raft_consensus.cc:1261] T b589127c2a45441bb78f306389d5ccf2 P 649129489c4049588da477db3e4fba75 [term 1 FOLLOWER]: Refusing update from remote peer 1fc5a333bcb14ce39f96fe2fef5c9f7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:02:40.980522 11906 raft_consensus.cc:1261] T b589127c2a45441bb78f306389d5ccf2 P a43d1b2c6d624cf698b6cd91b0e68335 [term 1 FOLLOWER]: Refusing update from remote peer 1fc5a333bcb14ce39f96fe2fef5c9f7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:02:40.981042 12049 consensus_queue.cc:1035] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:40.981359 12082 consensus_queue.cc:1035] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:41.025980 11470 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:02:41.026197 11470 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:02:41.029440 11470 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c: Sending DeleteTablet for 3 replicas of tablet b589127c2a45441bb78f306389d5ccf2
I20260926 15:02:41.030414 11887 tablet_service.cc:1514] Processing DeleteTablet for tablet b589127c2a45441bb78f306389d5ccf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:02:41 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:39884
I20260926 15:02:41.030702 12063 tablet_replica.cc:330] T b589127c2a45441bb78f306389d5ccf2 P a43d1b2c6d624cf698b6cd91b0e68335: stopping tablet replica
I20260926 15:02:41.030646 11606 tablet_service.cc:1514] Processing DeleteTablet for tablet b589127c2a45441bb78f306389d5ccf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:02:41 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:50718
I20260926 15:02:41.030887 12063 raft_consensus.cc:2229] T b589127c2a45441bb78f306389d5ccf2 P a43d1b2c6d624cf698b6cd91b0e68335 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:41.030916 12065 tablet_replica.cc:330] T b589127c2a45441bb78f306389d5ccf2 P 649129489c4049588da477db3e4fba75: stopping tablet replica
I20260926 15:02:41.031100 12063 raft_consensus.cc:2258] T b589127c2a45441bb78f306389d5ccf2 P a43d1b2c6d624cf698b6cd91b0e68335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:41.031103 12065 raft_consensus.cc:2229] T b589127c2a45441bb78f306389d5ccf2 P 649129489c4049588da477db3e4fba75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:41.031287 12065 raft_consensus.cc:2258] T b589127c2a45441bb78f306389d5ccf2 P 649129489c4049588da477db3e4fba75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:41.031575 11470 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48988:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260926 15:02:41.031798 11747 tablet_service.cc:1514] Processing DeleteTablet for tablet b589127c2a45441bb78f306389d5ccf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:02:41 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:47604
I20260926 15:02:41.032054 12064 tablet_replica.cc:330] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: stopping tablet replica
I20260926 15:02:41.032256 12064 raft_consensus.cc:2229] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:02:41.032410 12063 ts_tablet_manager.cc:1905] T b589127c2a45441bb78f306389d5ccf2 P a43d1b2c6d624cf698b6cd91b0e68335: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 15:02:41.032562 11470 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:02:41.032697 12065 ts_tablet_manager.cc:1905] T b589127c2a45441bb78f306389d5ccf2 P 649129489c4049588da477db3e4fba75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:41.032773 12064 raft_consensus.cc:2258] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:41.034142 12064 ts_tablet_manager.cc:1905] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:41.039086 12063 ts_tablet_manager.cc:1918] T b589127c2a45441bb78f306389d5ccf2 P a43d1b2c6d624cf698b6cd91b0e68335: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 15:02:41.039310 12063 log.cc:1196] T b589127c2a45441bb78f306389d5ccf2 P a43d1b2c6d624cf698b6cd91b0e68335: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/b589127c2a45441bb78f306389d5ccf2
I20260926 15:02:41.039327 12065 ts_tablet_manager.cc:1918] T b589127c2a45441bb78f306389d5ccf2 P 649129489c4049588da477db3e4fba75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 15:02:41.039695 12063 ts_tablet_manager.cc:1939] T b589127c2a45441bb78f306389d5ccf2 P a43d1b2c6d624cf698b6cd91b0e68335: Deleting consensus metadata
I20260926 15:02:41.039738 12065 log.cc:1196] T b589127c2a45441bb78f306389d5ccf2 P 649129489c4049588da477db3e4fba75: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-0/wal/wals/b589127c2a45441bb78f306389d5ccf2
I20260926 15:02:41.040126 12065 ts_tablet_manager.cc:1939] T b589127c2a45441bb78f306389d5ccf2 P 649129489c4049588da477db3e4fba75: Deleting consensus metadata
I20260926 15:02:41.041210 11455 catalog_manager.cc:4841] TS a43d1b2c6d624cf698b6cd91b0e68335 (127.9.123.67:42333): tablet b589127c2a45441bb78f306389d5ccf2 (table test-table [id=acdc4e0266a0418fb850b37151e8f869]) successfully deleted
I20260926 15:02:41.041448 11456 catalog_manager.cc:4841] TS 649129489c4049588da477db3e4fba75 (127.9.123.65:34481): tablet b589127c2a45441bb78f306389d5ccf2 (table test-table [id=acdc4e0266a0418fb850b37151e8f869]) successfully deleted
I20260926 15:02:41.043035 12064 ts_tablet_manager.cc:1918] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 15:02:41.043190 12064 log.cc:1196] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/b589127c2a45441bb78f306389d5ccf2
I20260926 15:02:41.043527 12064 ts_tablet_manager.cc:1939] T b589127c2a45441bb78f306389d5ccf2 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Deleting consensus metadata
I20260926 15:02:41.044514 11456 catalog_manager.cc:4841] TS 1fc5a333bcb14ce39f96fe2fef5c9f7a (127.9.123.66:45177): tablet b589127c2a45441bb78f306389d5ccf2 (table test-table [id=acdc4e0266a0418fb850b37151e8f869]) successfully deleted
I20260926 15:02:41.050277 11606 tablet_service.cc:1467] Processing CreateTablet for tablet 212229bd3baa42a6aff23aa54921bd65 (DEFAULT_TABLE table=test-table [id=e0a7ea193a8249179f42634cb3a0827a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:41.050386 11747 tablet_service.cc:1467] Processing CreateTablet for tablet 212229bd3baa42a6aff23aa54921bd65 (DEFAULT_TABLE table=test-table [id=e0a7ea193a8249179f42634cb3a0827a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:41.050570 11887 tablet_service.cc:1467] Processing CreateTablet for tablet 212229bd3baa42a6aff23aa54921bd65 (DEFAULT_TABLE table=test-table [id=e0a7ea193a8249179f42634cb3a0827a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:41.050741 11606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 212229bd3baa42a6aff23aa54921bd65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:41.050863 11747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 212229bd3baa42a6aff23aa54921bd65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:41.050933 11887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 212229bd3baa42a6aff23aa54921bd65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:41.058117 12074 tablet_bootstrap.cc:492] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335: Bootstrap starting.
I20260926 15:02:41.058827 12075 tablet_bootstrap.cc:492] T 212229bd3baa42a6aff23aa54921bd65 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Bootstrap starting.
I20260926 15:02:41.059810 12074 tablet_bootstrap.cc:654] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:41.060593 12075 tablet_bootstrap.cc:654] T 212229bd3baa42a6aff23aa54921bd65 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:41.061735 12076 tablet_bootstrap.cc:492] T 212229bd3baa42a6aff23aa54921bd65 P 649129489c4049588da477db3e4fba75: Bootstrap starting.
I20260926 15:02:41.063433 12076 tablet_bootstrap.cc:654] T 212229bd3baa42a6aff23aa54921bd65 P 649129489c4049588da477db3e4fba75: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:41.065680 12076 tablet_bootstrap.cc:492] T 212229bd3baa42a6aff23aa54921bd65 P 649129489c4049588da477db3e4fba75: No bootstrap required, opened a new log
I20260926 15:02:41.065945 12076 ts_tablet_manager.cc:1397] T 212229bd3baa42a6aff23aa54921bd65 P 649129489c4049588da477db3e4fba75: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260926 15:02:41.066826 12076 raft_consensus.cc:348] T 212229bd3baa42a6aff23aa54921bd65 P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:41.067157 12076 raft_consensus.cc:374] T 212229bd3baa42a6aff23aa54921bd65 P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:41.067256 12076 raft_consensus.cc:729] T 212229bd3baa42a6aff23aa54921bd65 P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 649129489c4049588da477db3e4fba75, State: Initialized, Role: FOLLOWER
I20260926 15:02:41.067627 12076 consensus_queue.cc:260] T 212229bd3baa42a6aff23aa54921bd65 P 649129489c4049588da477db3e4fba75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:41.068266 12076 ts_tablet_manager.cc:1428] T 212229bd3baa42a6aff23aa54921bd65 P 649129489c4049588da477db3e4fba75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260926 15:02:41.069897 12075 tablet_bootstrap.cc:492] T 212229bd3baa42a6aff23aa54921bd65 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: No bootstrap required, opened a new log
I20260926 15:02:41.070127 12075 ts_tablet_manager.cc:1397] T 212229bd3baa42a6aff23aa54921bd65 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20260926 15:02:41.070717 12074 tablet_bootstrap.cc:492] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335: No bootstrap required, opened a new log
I20260926 15:02:41.070894 12074 ts_tablet_manager.cc:1397] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260926 15:02:41.070933 12075 raft_consensus.cc:348] T 212229bd3baa42a6aff23aa54921bd65 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:41.071260 12075 raft_consensus.cc:374] T 212229bd3baa42a6aff23aa54921bd65 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:41.071358 12075 raft_consensus.cc:729] T 212229bd3baa42a6aff23aa54921bd65 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc5a333bcb14ce39f96fe2fef5c9f7a, State: Initialized, Role: FOLLOWER
I20260926 15:02:41.071641 12075 consensus_queue.cc:260] T 212229bd3baa42a6aff23aa54921bd65 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:41.071705 12074 raft_consensus.cc:348] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:41.071915 12074 raft_consensus.cc:374] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:41.071982 12074 raft_consensus.cc:729] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a43d1b2c6d624cf698b6cd91b0e68335, State: Initialized, Role: FOLLOWER
I20260926 15:02:41.072294 12074 consensus_queue.cc:260] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:41.073762 12075 ts_tablet_manager.cc:1428] T 212229bd3baa42a6aff23aa54921bd65 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260926 15:02:41.074186 12074 ts_tablet_manager.cc:1428] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260926 15:02:41.297360 12079 raft_consensus.cc:482] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:41.297572 12079 raft_consensus.cc:504] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:41.299134 12079 leader_election.cc:288] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 649129489c4049588da477db3e4fba75 (127.9.123.65:34481), 1fc5a333bcb14ce39f96fe2fef5c9f7a (127.9.123.66:45177)
I20260926 15:02:41.307566 12095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:41.299541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:51421 (local address 127.9.123.65:34481)
0926 15:02:41.301058 (+ 1517us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:41.301064 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:41.301082 (+ 18us) server_negotiation.cc:407] Connection header received
0926 15:02:41.301177 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:41.301199 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:41.301284 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:41.301375 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:41.301999 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:41.303087 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:41.305392 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:41.306468 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:41.306672 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:41.306749 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:41.307118 (+ 369us) server_negotiation.cc:299] Negotiation successful
0926 15:02:41.307263 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1383,"thread_start_us":1287,"threads_started":1}
I20260926 15:02:41.307652 12084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:41.300375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:34285 (local address 127.9.123.66:45177)
0926 15:02:41.300551 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:41.300557 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:41.300719 (+ 162us) server_negotiation.cc:407] Connection header received
0926 15:02:41.300796 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:41.300802 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:41.300875 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:41.300957 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:41.301654 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:41.302524 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:41.304804 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:41.306662 (+ 1858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:41.306793 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:41.306860 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:41.307224 (+ 364us) server_negotiation.cc:299] Negotiation successful
0926 15:02:41.307360 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 15:02:41.307612 12094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:41.299390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:34481 (local address 127.9.123.67:51421)
0926 15:02:41.299901 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:41.299920 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:41.300064 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:41.301414 (+ 1350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:41.301418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:41.301449 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:41.301805 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:41.301816 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:41.303223 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:41.303227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:41.305208 (+ 1981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:41.305223 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:41.306700 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:41.306707 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:41.306923 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:41.306962 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 15:02:41.307076 (+ 114us) client_negotiation.cc:240] Negotiation successful
0926 15:02:41.307225 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":23,"thread_start_us":235,"threads_started":1}
I20260926 15:02:41.308380 12096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:41.300035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:45177 (local address 127.9.123.67:34285)
0926 15:02:41.300535 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:41.300551 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:41.300663 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:41.301011 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:41.301027 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:41.301060 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:41.301463 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:41.301477 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:41.302653 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:41.302659 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:41.304600 (+ 1941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:41.304616 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:41.306909 (+ 2293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:41.306915 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:41.307047 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:41.307078 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 15:02:41.307857 (+ 779us) client_negotiation.cc:240] Negotiation successful
0926 15:02:41.307978 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":3,"thread_start_us":521,"threads_started":1}
I20260926 15:02:41.308698 11627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "212229bd3baa42a6aff23aa54921bd65" candidate_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "649129489c4049588da477db3e4fba75" is_pre_election: true
I20260926 15:02:41.308956 11627 raft_consensus.cc:2454] T 212229bd3baa42a6aff23aa54921bd65 P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a43d1b2c6d624cf698b6cd91b0e68335 in term 0.
I20260926 15:02:41.308951 11766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "212229bd3baa42a6aff23aa54921bd65" candidate_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" is_pre_election: true
I20260926 15:02:41.309219 11766 raft_consensus.cc:2454] T 212229bd3baa42a6aff23aa54921bd65 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a43d1b2c6d624cf698b6cd91b0e68335 in term 0.
I20260926 15:02:41.309499 11839 leader_election.cc:302] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 649129489c4049588da477db3e4fba75, a43d1b2c6d624cf698b6cd91b0e68335; no voters:
I20260926 15:02:41.309926 12079 raft_consensus.cc:2781] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:41.310039 12079 raft_consensus.cc:482] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:41.310132 12079 raft_consensus.cc:3037] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:41.312240 12079 raft_consensus.cc:504] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:41.312796 12079 leader_election.cc:288] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335 [CANDIDATE]: Term 1 election: Requested vote from peers 649129489c4049588da477db3e4fba75 (127.9.123.65:34481), 1fc5a333bcb14ce39f96fe2fef5c9f7a (127.9.123.66:45177)
I20260926 15:02:41.313462 11627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "212229bd3baa42a6aff23aa54921bd65" candidate_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "649129489c4049588da477db3e4fba75"
I20260926 15:02:41.313522 11766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "212229bd3baa42a6aff23aa54921bd65" candidate_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a"
I20260926 15:02:41.313644 11627 raft_consensus.cc:3037] T 212229bd3baa42a6aff23aa54921bd65 P 649129489c4049588da477db3e4fba75 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:41.313740 11766 raft_consensus.cc:3037] T 212229bd3baa42a6aff23aa54921bd65 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:41.315543 11627 raft_consensus.cc:2454] T 212229bd3baa42a6aff23aa54921bd65 P 649129489c4049588da477db3e4fba75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a43d1b2c6d624cf698b6cd91b0e68335 in term 1.
I20260926 15:02:41.315994 11839 leader_election.cc:302] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 649129489c4049588da477db3e4fba75, a43d1b2c6d624cf698b6cd91b0e68335; no voters:
I20260926 15:02:41.316258 12079 raft_consensus.cc:2781] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:41.316722 12079 raft_consensus.cc:686] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335 [term 1 LEADER]: Becoming Leader. State: Replica: a43d1b2c6d624cf698b6cd91b0e68335, State: Running, Role: LEADER
I20260926 15:02:41.317231 11766 raft_consensus.cc:2454] T 212229bd3baa42a6aff23aa54921bd65 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a43d1b2c6d624cf698b6cd91b0e68335 in term 1.
I20260926 15:02:41.317173 12079 consensus_queue.cc:237] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } }
I20260926 15:02:41.320626 11470 catalog_manager.cc:5495] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335 reported cstate change: term changed from 0 to 1, leader changed from <none> to a43d1b2c6d624cf698b6cd91b0e68335 (127.9.123.67). New cstate: current_term: 1 leader_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a43d1b2c6d624cf698b6cd91b0e68335" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42333 } health_report { overall_health: HEALTHY } } }
I20260926 15:02:41.382822 12099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:41.375747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59752 (local address 127.9.123.67:42333)
0926 15:02:41.376345 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:41.376349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:41.376369 (+ 20us) server_negotiation.cc:407] Connection header received
0926 15:02:41.376450 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:41.376453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:41.376536 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:41.376671 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:02:41.377171 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:41.378360 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:41.380235 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:41.380673 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:41.380771 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:41.381425 (+ 654us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:02:41.382072 (+ 647us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:02:41.382188 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:41.382384 (+ 196us) server_negotiation.cc:299] Negotiation successful
0926 15:02:41.382461 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":357,"threads_started":1}
I20260926 15:02:41.399489 12084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:41.393827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:51147 (local address 127.9.123.66:45177)
0926 15:02:41.394035 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:41.394042 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:41.394664 (+ 622us) server_negotiation.cc:407] Connection header received
0926 15:02:41.394856 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:41.394862 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:41.394931 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:41.395007 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:02:41.395482 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:41.396268 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:41.397762 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:41.398224 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:41.398350 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:41.398774 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:02:41.398907 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:02:41.399005 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:41.399139 (+ 134us) server_negotiation.cc:299] Negotiation successful
0926 15:02:41.399217 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 15:02:41.399614 12102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:41.393665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:45177 (local address 127.9.123.67:51147)
0926 15:02:41.394550 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:41.394575 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:41.394719 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:41.395035 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:41.395039 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:41.395051 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 15:02:41.395347 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:41.395355 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:41.396398 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:41.396404 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:41.397621 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:41.397635 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:41.398392 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:41.398398 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:41.398534 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:41.398601 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 15:02:41.399022 (+ 421us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 15:02:41.399034 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 15:02:41.399109 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 15:02:41.399256 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"thread_start_us":527,"threads_started":1}
I20260926 15:02:41.408660 11766 raft_consensus.cc:1261] T 212229bd3baa42a6aff23aa54921bd65 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 1 FOLLOWER]: Refusing update from remote peer a43d1b2c6d624cf698b6cd91b0e68335: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:02:41.408710 11627 raft_consensus.cc:1261] T 212229bd3baa42a6aff23aa54921bd65 P 649129489c4049588da477db3e4fba75 [term 1 FOLLOWER]: Refusing update from remote peer a43d1b2c6d624cf698b6cd91b0e68335: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:02:41.409534 12097 consensus_queue.cc:1035] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc5a333bcb14ce39f96fe2fef5c9f7a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:41.419672 12079 consensus_queue.cc:1035] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "649129489c4049588da477db3e4fba75" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:41.451138 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:41.444705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:42333 (local address 127.9.123.66:46285)
0926 15:02:41.445013 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:41.445033 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:41.445172 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:41.445686 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:41.445691 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:41.445704 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 15:02:41.445886 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:41.445897 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:41.447418 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:41.447424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:41.448919 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:41.448931 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:41.449925 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:41.449931 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:41.450125 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:41.450174 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 15:02:41.450565 (+ 391us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 15:02:41.450576 (+ 11us) client_negotiation.cc:769] Sending connection context
0926 15:02:41.450640 (+ 64us) client_negotiation.cc:240] Negotiation successful
0926 15:02:41.450787 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166}
I20260926 15:02:41.451833 12099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:41.444696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:46285 (local address 127.9.123.67:42333)
0926 15:02:41.444849 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:41.444854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:41.445089 (+ 235us) server_negotiation.cc:407] Connection header received
0926 15:02:41.445332 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:41.445338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:41.445421 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:41.445509 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:02:41.446035 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:41.447246 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:41.449053 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:41.449702 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:41.449813 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:41.450330 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:02:41.450458 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:02:41.450525 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:41.451483 (+ 958us) server_negotiation.cc:299] Negotiation successful
0926 15:02:41.451568 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260926 15:02:41.452834 12045 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260926 15:02:43.048521 11470 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:02:43.048735 11470 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:02:43.053123 11470 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff517d6e5a6a4201a3b322b4fbe93c2c: Sending DeleteTablet for 3 replicas of tablet 212229bd3baa42a6aff23aa54921bd65
I20260926 15:02:43.054046 11606 tablet_service.cc:1514] Processing DeleteTablet for tablet 212229bd3baa42a6aff23aa54921bd65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:02:43 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:50718
I20260926 15:02:43.054057 11747 tablet_service.cc:1514] Processing DeleteTablet for tablet 212229bd3baa42a6aff23aa54921bd65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:02:43 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:47604
I20260926 15:02:43.054123 11887 tablet_service.cc:1514] Processing DeleteTablet for tablet 212229bd3baa42a6aff23aa54921bd65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:02:43 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:39884
I20260926 15:02:43.054559 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 11541
I20260926 15:02:43.069455 12130 tablet_replica.cc:330] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335: stopping tablet replica
I20260926 15:02:43.069850 12130 raft_consensus.cc:2229] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335 [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:02:43.070165 12129 tablet_replica.cc:330] T 212229bd3baa42a6aff23aa54921bd65 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: stopping tablet replica
I20260926 15:02:43.070333 12131 tablet_replica.cc:330] T 212229bd3baa42a6aff23aa54921bd65 P 649129489c4049588da477db3e4fba75: stopping tablet replica
I20260926 15:02:43.070475 12129 raft_consensus.cc:2229] T 212229bd3baa42a6aff23aa54921bd65 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:43.070534 12131 raft_consensus.cc:2229] T 212229bd3baa42a6aff23aa54921bd65 P 649129489c4049588da477db3e4fba75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:43.070675 12130 raft_consensus.cc:2258] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:43.070747 12129 raft_consensus.cc:2258] T 212229bd3baa42a6aff23aa54921bd65 P 1fc5a333bcb14ce39f96fe2fef5c9f7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:43.070748 12131 raft_consensus.cc:2258] T 212229bd3baa42a6aff23aa54921bd65 P 649129489c4049588da477db3e4fba75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:43.071655 12130 ts_tablet_manager.cc:1905] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:43.071995 12129 ts_tablet_manager.cc:1905] T 212229bd3baa42a6aff23aa54921bd65 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:43.072167 12131 ts_tablet_manager.cc:1905] T 212229bd3baa42a6aff23aa54921bd65 P 649129489c4049588da477db3e4fba75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:43.078531 12130 ts_tablet_manager.cc:1918] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 15:02:43.078537 12131 ts_tablet_manager.cc:1918] T 212229bd3baa42a6aff23aa54921bd65 P 649129489c4049588da477db3e4fba75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 15:02:43.078683 12131 log.cc:1196] T 212229bd3baa42a6aff23aa54921bd65 P 649129489c4049588da477db3e4fba75: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-0/wal/wals/212229bd3baa42a6aff23aa54921bd65
I20260926 15:02:43.078701 12129 ts_tablet_manager.cc:1918] T 212229bd3baa42a6aff23aa54921bd65 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260926 15:02:43.078775 12130 log.cc:1196] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/212229bd3baa42a6aff23aa54921bd65
I20260926 15:02:43.078970 12129 log.cc:1196] T 212229bd3baa42a6aff23aa54921bd65 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/212229bd3baa42a6aff23aa54921bd65
Sep 26 15:02:43 dist-test-slave-7153 krb5kdc[11425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434960, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.123.65@KRBTEST.COM
I20260926 15:02:43.079178 12131 ts_tablet_manager.cc:1939] T 212229bd3baa42a6aff23aa54921bd65 P 649129489c4049588da477db3e4fba75: Deleting consensus metadata
I20260926 15:02:43.079178 12130 ts_tablet_manager.cc:1939] T 212229bd3baa42a6aff23aa54921bd65 P a43d1b2c6d624cf698b6cd91b0e68335: Deleting consensus metadata
I20260926 15:02:43.079445 12129 ts_tablet_manager.cc:1939] T 212229bd3baa42a6aff23aa54921bd65 P 1fc5a333bcb14ce39f96fe2fef5c9f7a: Deleting consensus metadata
I20260926 15:02:43.080315 11455 catalog_manager.cc:4841] TS a43d1b2c6d624cf698b6cd91b0e68335 (127.9.123.67:42333): tablet 212229bd3baa42a6aff23aa54921bd65 (table test-table [id=e0a7ea193a8249179f42634cb3a0827a]) successfully deleted
I20260926 15:02:43.080315 11456 catalog_manager.cc:4841] TS 649129489c4049588da477db3e4fba75 (127.9.123.65:34481): tablet 212229bd3baa42a6aff23aa54921bd65 (table test-table [id=e0a7ea193a8249179f42634cb3a0827a]) successfully deleted
I20260926 15:02:43.080610 11456 catalog_manager.cc:4841] TS 1fc5a333bcb14ce39f96fe2fef5c9f7a (127.9.123.66:45177): tablet 212229bd3baa42a6aff23aa54921bd65 (table test-table [id=e0a7ea193a8249179f42634cb3a0827a]) successfully deleted
I20260926 15:02:43.086396 12133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:43.057075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55324 (local address 127.9.123.65:34481)
0926 15:02:43.062814 (+ 5739us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:43.062820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:43.069005 (+ 6185us) server_negotiation.cc:407] Connection header received
0926 15:02:43.069288 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:43.069294 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:43.069387 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:43.069489 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:43.071319 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:43.072340 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:43.074256 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:43.074907 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:43.075003 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:43.080321 (+ 5318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:43.080341 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:43.080365 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:43.080408 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:43.083555 (+ 3147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:43.084200 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:43.084206 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:43.084208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:43.084320 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:43.084757 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:43.084762 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:43.084766 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:43.085229 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:43.085528 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:43.085803 (+ 275us) server_negotiation.cc:299] Negotiation successful
0926 15:02:43.086018 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5604,"thread_start_us":5468,"threads_started":1}
W20260926 15:02:43.087139 11672 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55324
W20260926 15:02:43.087862 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 11541: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 15:02:43.088111 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 11541
I20260926 15:02:43.106701 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 11681
Sep 26 15:02:43 dist-test-slave-7153 krb5kdc[11425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434960, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.123.66@KRBTEST.COM
I20260926 15:02:43.121593 12134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:43.107487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35882 (local address 127.9.123.66:45177)
0926 15:02:43.107955 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:43.107959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:43.107977 (+ 18us) server_negotiation.cc:407] Connection header received
0926 15:02:43.108031 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:43.108034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:43.108110 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:43.108201 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:43.109473 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:43.110238 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:43.111563 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:43.111980 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:43.112085 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:43.116077 (+ 3992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:43.116099 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:43.116121 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:43.116183 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:43.119022 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:43.119658 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:43.119665 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:43.119667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:43.119820 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:43.120138 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:43.120142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:43.120145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:43.120496 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:43.120724 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:43.120952 (+ 228us) server_negotiation.cc:299] Negotiation successful
0926 15:02:43.121333 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":245,"threads_started":1}
W20260926 15:02:43.122215 11812 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35882
W20260926 15:02:43.122800 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 11681: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 15:02:43.122943 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 11681
I20260926 15:02:43.143615 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 11821
Sep 26 15:02:43 dist-test-slave-7153 krb5kdc[11425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434960, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.123.67@KRBTEST.COM
I20260926 15:02:43.157963 12135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:43.144255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59758 (local address 127.9.123.67:42333)
0926 15:02:43.144732 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:43.144736 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:43.144751 (+ 15us) server_negotiation.cc:407] Connection header received
0926 15:02:43.144802 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:43.144805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:43.144859 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:43.144944 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:43.146204 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:43.146973 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:43.148138 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:43.148569 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:43.148663 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:43.152681 (+ 4018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:43.152696 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:43.152711 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:43.152746 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:43.155348 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:43.155988 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:43.155995 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:43.155997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:43.156127 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:43.156475 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:43.156478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:43.156481 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:43.156830 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:43.157130 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:43.157436 (+ 306us) server_negotiation.cc:299] Negotiation successful
0926 15:02:43.157653 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":260,"threads_started":1}
W20260926 15:02:43.158600 11952 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59758
W20260926 15:02:43.159188 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 11821: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 15:02:43.159332 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 11821
I20260926 15:02:43.175976 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 11438
I20260926 15:02:43.177239 11510 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:02:43.289458 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 11438
2026-09-26T15:02:43Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (11117 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:02:43 dist-test-slave-7153 krb5kdc[12139](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:02:43 dist-test-slave-7153 krb5kdc[12139](info): set up 2 sockets
Sep 26 15:02:43 dist-test-slave-7153 krb5kdc[12139](info): commencing operation
krb5kdc: starting...
W20260926 15:02:44.293207 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.964s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:02:44 dist-test-slave-7153 krb5kdc[12139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:02:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:02:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:02:44.476989 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:43511
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:45431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:43511
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:44.794601 12152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:44.794922 12152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:44.795013 12152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:44.802325 12152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:02:44.802455 12152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:44.802500 12152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:44.802556 12152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:02:44.802591 12152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:02:44.812497 12152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:45431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:43511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:43511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:44.813869 12152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:44.815130 12152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:44.828379 12157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:44.828238 12158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:44.828488 12160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:44.828879 12152 server_base.cc:1034] running on GCE node
I20260926 15:02:44.829793 12152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:44.831379 12152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:44.832657 12152 hybrid_clock.cc:648] HybridClock initialized: now 1790434964832596 us; error 51 us; skew 500 ppm
I20260926 15:02:44.833063 12152 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:44 dist-test-slave-7153 krb5kdc[12139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434964, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:44.836551 12152 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:02:44.838340 12152 webserver.cc:460] Webserver started at http://127.9.123.126:45137/ using document root <none> and password file <none>
I20260926 15:02:44.839097 12152 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:44.839226 12152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:44.839536 12152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:44.842777 12152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "a96a54a9e76844fc9df8069f96683911"
format_stamp: "Formatted at 2026-09-26 15:02:44 on dist-test-slave-7153"
server_key: "d1739a060676308efac9fa0c9e18c3c0"
server_key_iv: "f97edc85d75e533dadbc991b617cbacf"
server_key_version: "encryptionkey@0"
I20260926 15:02:44.843693 12152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "a96a54a9e76844fc9df8069f96683911"
format_stamp: "Formatted at 2026-09-26 15:02:44 on dist-test-slave-7153"
server_key: "d1739a060676308efac9fa0c9e18c3c0"
server_key_iv: "f97edc85d75e533dadbc991b617cbacf"
server_key_version: "encryptionkey@0"
I20260926 15:02:44.849359 12152 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 15:02:44.853391 12167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:44.855057 12152 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:02:44.855263 12152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "a96a54a9e76844fc9df8069f96683911"
format_stamp: "Formatted at 2026-09-26 15:02:44 on dist-test-slave-7153"
server_key: "d1739a060676308efac9fa0c9e18c3c0"
server_key_iv: "f97edc85d75e533dadbc991b617cbacf"
server_key_version: "encryptionkey@0"
I20260926 15:02:44.855470 12152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:44.876720 12152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:44.881342 12152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:44.881635 12152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:44.898917 12152 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:43511
I20260926 15:02:44.898937 12218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:43511 every 8 connection(s)
I20260926 15:02:44.900254 12152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:02:44.904189 12219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:44.904887 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 12152
I20260926 15:02:44.905081 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:02:44.905706 9709 external_mini_cluster.cc:1428] Setting key fb59b02c2c5c1aa4d0e3d026b432e9ea
I20260926 15:02:44.918376 12219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a96a54a9e76844fc9df8069f96683911: Bootstrap starting.
Sep 26 15:02:44 dist-test-slave-7153 krb5kdc[12139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:44.922338 12219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a96a54a9e76844fc9df8069f96683911: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:44.923434 12219 log.cc:824] T 00000000000000000000000000000000 P a96a54a9e76844fc9df8069f96683911: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:44.926779 12219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a96a54a9e76844fc9df8069f96683911: No bootstrap required, opened a new log
I20260926 15:02:44.930583 12222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:44.907667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32806 (local address 127.9.123.126:43511)
0926 15:02:44.908432 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:44.908446 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:44.908505 (+ 59us) server_negotiation.cc:407] Connection header received
0926 15:02:44.910269 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:44.910303 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:44.910792 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:44.911173 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:44.913056 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:44.914521 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:44.916334 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:44.917064 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:44.917247 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:44.921948 (+ 4701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:44.921957 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:44.921974 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:44.922018 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:44.925344 (+ 3326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:44.926121 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:44.926131 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:44.926139 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:44.926352 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:44.926718 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:44.926723 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:44.926725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:44.927442 (+ 717us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:44.927766 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:44.928530 (+ 764us) server_negotiation.cc:299] Negotiation successful
0926 15:02:44.929024 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":330,"threads_started":1}
I20260926 15:02:44.936050 12219 raft_consensus.cc:348] T 00000000000000000000000000000000 P a96a54a9e76844fc9df8069f96683911 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a96a54a9e76844fc9df8069f96683911" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 43511 } }
I20260926 15:02:44.936409 12219 raft_consensus.cc:374] T 00000000000000000000000000000000 P a96a54a9e76844fc9df8069f96683911 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:44.936480 12219 raft_consensus.cc:729] T 00000000000000000000000000000000 P a96a54a9e76844fc9df8069f96683911 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a96a54a9e76844fc9df8069f96683911, State: Initialized, Role: FOLLOWER
I20260926 15:02:44.937019 12219 consensus_queue.cc:260] T 00000000000000000000000000000000 P a96a54a9e76844fc9df8069f96683911 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a96a54a9e76844fc9df8069f96683911" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 43511 } }
I20260926 15:02:44.937286 12219 raft_consensus.cc:388] T 00000000000000000000000000000000 P a96a54a9e76844fc9df8069f96683911 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:02:44.937417 12219 raft_consensus.cc:482] T 00000000000000000000000000000000 P a96a54a9e76844fc9df8069f96683911 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:02:44.937631 12219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a96a54a9e76844fc9df8069f96683911 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:44.940214 12219 raft_consensus.cc:504] T 00000000000000000000000000000000 P a96a54a9e76844fc9df8069f96683911 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a96a54a9e76844fc9df8069f96683911" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 43511 } }
I20260926 15:02:44.940701 12219 leader_election.cc:302] T 00000000000000000000000000000000 P a96a54a9e76844fc9df8069f96683911 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a96a54a9e76844fc9df8069f96683911; no voters:
I20260926 15:02:44.941299 12219 leader_election.cc:288] T 00000000000000000000000000000000 P a96a54a9e76844fc9df8069f96683911 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:02:44.941406 12224 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a96a54a9e76844fc9df8069f96683911 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:44.942039 12224 raft_consensus.cc:686] T 00000000000000000000000000000000 P a96a54a9e76844fc9df8069f96683911 [term 1 LEADER]: Becoming Leader. State: Replica: a96a54a9e76844fc9df8069f96683911, State: Running, Role: LEADER
I20260926 15:02:44.942633 12224 consensus_queue.cc:237] T 00000000000000000000000000000000 P a96a54a9e76844fc9df8069f96683911 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a96a54a9e76844fc9df8069f96683911" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 43511 } }
I20260926 15:02:44.943404 12219 sys_catalog.cc:564] T 00000000000000000000000000000000 P a96a54a9e76844fc9df8069f96683911 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:02:44.945696 12225 sys_catalog.cc:455] T 00000000000000000000000000000000 P a96a54a9e76844fc9df8069f96683911 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a96a54a9e76844fc9df8069f96683911" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a96a54a9e76844fc9df8069f96683911" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 43511 } } }
I20260926 15:02:44.945999 12225 sys_catalog.cc:458] T 00000000000000000000000000000000 P a96a54a9e76844fc9df8069f96683911 [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:44.946837 12231 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:02:44.946628 12226 sys_catalog.cc:455] T 00000000000000000000000000000000 P a96a54a9e76844fc9df8069f96683911 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a96a54a9e76844fc9df8069f96683911. Latest consensus state: current_term: 1 leader_uuid: "a96a54a9e76844fc9df8069f96683911" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a96a54a9e76844fc9df8069f96683911" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 43511 } } }
I20260926 15:02:44.948654 12226 sys_catalog.cc:458] T 00000000000000000000000000000000 P a96a54a9e76844fc9df8069f96683911 [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:44.952883 12231 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:02:44.962628 12231 catalog_manager.cc:1347] Generated new cluster ID: 0629a3a912aa47bdb07941b19fbcfe6c
I20260926 15:02:44.962755 12231 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:02:44.970467 12231 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:02:44.972573 12231 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:02:44.982184 12231 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a96a54a9e76844fc9df8069f96683911: Generated new TSK 0
I20260926 15:02:44.983032 12231 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.123.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
I20260926 15:02:45.041971 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:43511
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:45431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:45.389187 12245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:45.389505 12245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:45.389608 12245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:45.397426 12245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:45.397575 12245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:45.397724 12245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:02:45.407577 12245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:45431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:43511
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:45.408883 12245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:45.410223 12245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:45.422760 12253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:45.422695 12251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:45.422791 12250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:45.423945 12245 server_base.cc:1034] running on GCE node
I20260926 15:02:45.424566 12245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:45.425576 12245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:45.426815 12245 hybrid_clock.cc:648] HybridClock initialized: now 1790434965426748 us; error 53 us; skew 500 ppm
I20260926 15:02:45.427127 12245 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:45 dist-test-slave-7153 krb5kdc[12139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434965, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:45.430426 12245 init.cc:377] Logged in from keytab as kudu/127.9.123.65@KRBTEST.COM (short username kudu)
I20260926 15:02:45.431975 12245 webserver.cc:460] Webserver started at http://127.9.123.65:32769/ using document root <none> and password file <none>
I20260926 15:02:45.432663 12245 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:45.432778 12245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:45.433076 12245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:45.436084 12245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "ed547f66f724445f967350549f327ec2"
format_stamp: "Formatted at 2026-09-26 15:02:45 on dist-test-slave-7153"
server_key: "3948b918c7035feef8384a2472ecfb89"
server_key_iv: "d7ee458e36c9cd7279b831a9184c6b28"
server_key_version: "encryptionkey@0"
I20260926 15:02:45.436908 12245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "ed547f66f724445f967350549f327ec2"
format_stamp: "Formatted at 2026-09-26 15:02:45 on dist-test-slave-7153"
server_key: "3948b918c7035feef8384a2472ecfb89"
server_key_iv: "d7ee458e36c9cd7279b831a9184c6b28"
server_key_version: "encryptionkey@0"
I20260926 15:02:45.442582 12245 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 15:02:45.446652 12260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:45.448263 12245 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:02:45.448434 12245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "ed547f66f724445f967350549f327ec2"
format_stamp: "Formatted at 2026-09-26 15:02:45 on dist-test-slave-7153"
server_key: "3948b918c7035feef8384a2472ecfb89"
server_key_iv: "d7ee458e36c9cd7279b831a9184c6b28"
server_key_version: "encryptionkey@0"
I20260926 15:02:45.448652 12245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:45.460664 12245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:45.464890 12245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:45.465209 12245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:45.466182 12245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:02:45.467603 12245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:45.467680 12245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:45.467795 12245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:45.467851 12245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:45.499290 12245 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:39677
I20260926 15:02:45.499372 12372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:39677 every 8 connection(s)
I20260926 15:02:45.500653 12245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:02:45.503460 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 12245
I20260926 15:02:45.503671 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:02:45.504451 9709 external_mini_cluster.cc:1428] Setting key 13629332ed2975c4d212600e58c6d1a3
Sep 26 15:02:45 dist-test-slave-7153 krb5kdc[12139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434965, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.123.66@KRBTEST.COM; defaulting to no policy
I20260926 15:02:45.521626 12375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:45.504254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:49519 (local address 127.9.123.126:43511)
0926 15:02:45.504708 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:45.504712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:45.505495 (+ 783us) server_negotiation.cc:407] Connection header received
0926 15:02:45.507647 (+ 2152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:45.507652 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:45.507733 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:45.507827 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:45.509459 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:45.510181 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:45.511748 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:45.512164 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:45.512278 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:45.516056 (+ 3778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:45.516062 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:45.516066 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:45.516101 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:45.518565 (+ 2464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:45.519336 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:45.519340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:45.519342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:45.519462 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:45.519931 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:45.519934 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:45.519937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:45.520208 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:45.520322 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:45.521202 (+ 880us) server_negotiation.cc:299] Negotiation successful
0926 15:02:45.521375 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":239,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.66@KRBTEST.COM" created.
I20260926 15:02:45.522652 12376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:45.504259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:43511 (local address 127.9.123.65:49519)
0926 15:02:45.505319 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:45.505370 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:45.507360 (+ 1990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:45.508100 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:45.508109 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:45.508640 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:45.509295 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:45.509315 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:45.510320 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:45.510324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:45.511614 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:45.511626 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:45.512390 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:45.512397 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:45.512635 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:45.513372 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:45.513396 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:45.515780 (+ 2384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:45.518725 (+ 2945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:45.518735 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:45.518757 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:45.519203 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:45.519573 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:45.519578 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:45.519583 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:45.519773 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:45.520338 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:45.520345 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:45.520751 (+ 406us) client_negotiation.cc:769] Sending connection context
0926 15:02:45.521029 (+ 278us) client_negotiation.cc:240] Negotiation successful
0926 15:02:45.521476 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":497,"threads_started":1}
I20260926 15:02:45.524471 12373 heartbeater.cc:347] Connected to a master server at 127.9.123.126:43511
I20260926 15:02:45.524852 12373 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:45.525847 12373 heartbeater.cc:511] Master 127.9.123.126:43511 requested a full tablet report, sending...
I20260926 15:02:45.528738 12184 ts_manager.cc:194] Registered new tserver with Master: ed547f66f724445f967350549f327ec2 (127.9.123.65:39677)
I20260926 15:02:45.530783 12184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.65@KRBTEST.COM'} at 127.9.123.65:49519
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
I20260926 15:02:45.548707 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:43511
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:45431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:45.864451 12379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:45.864774 12379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:45.864897 12379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:45.872195 12379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:45.872339 12379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:45.872478 12379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:02:45.882225 12379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:45431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:43511
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:45.883512 12379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:45.884804 12379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:45.898303 12387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:45.898233 12385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:45.898262 12384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:45.906584 12379 server_base.cc:1034] running on GCE node
I20260926 15:02:45.907299 12379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:45.908392 12379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:45.909615 12379 hybrid_clock.cc:648] HybridClock initialized: now 1790434965909557 us; error 54 us; skew 500 ppm
I20260926 15:02:45.909943 12379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:45 dist-test-slave-7153 krb5kdc[12139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434965, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:45.917892 12379 init.cc:377] Logged in from keytab as kudu/127.9.123.66@KRBTEST.COM (short username kudu)
I20260926 15:02:45.919576 12379 webserver.cc:460] Webserver started at http://127.9.123.66:41239/ using document root <none> and password file <none>
I20260926 15:02:45.920331 12379 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:45.920445 12379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:45.920804 12379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:45.923831 12379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "eaf3968d07cb43ffae74accb20328a88"
format_stamp: "Formatted at 2026-09-26 15:02:45 on dist-test-slave-7153"
server_key: "cbee2997e45a9e90ada56b0ac8e6c554"
server_key_iv: "b5e92a3a3b2cbeca2b2c8f29808ef6ea"
server_key_version: "encryptionkey@0"
I20260926 15:02:45.924607 12379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "eaf3968d07cb43ffae74accb20328a88"
format_stamp: "Formatted at 2026-09-26 15:02:45 on dist-test-slave-7153"
server_key: "cbee2997e45a9e90ada56b0ac8e6c554"
server_key_iv: "b5e92a3a3b2cbeca2b2c8f29808ef6ea"
server_key_version: "encryptionkey@0"
I20260926 15:02:45.929962 12379 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260926 15:02:45.933660 12394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:45.935113 12379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260926 15:02:45.935274 12379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "eaf3968d07cb43ffae74accb20328a88"
format_stamp: "Formatted at 2026-09-26 15:02:45 on dist-test-slave-7153"
server_key: "cbee2997e45a9e90ada56b0ac8e6c554"
server_key_iv: "b5e92a3a3b2cbeca2b2c8f29808ef6ea"
server_key_version: "encryptionkey@0"
I20260926 15:02:45.935472 12379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:45.958012 12379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:45.962162 12379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:45.962420 12379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:45.963366 12379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:02:45.964646 12379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:45.964720 12379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:45.964828 12379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:45.964874 12379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:45.993582 12379 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:39081
I20260926 15:02:45.993630 12506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:39081 every 8 connection(s)
I20260926 15:02:45.994889 12379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:02:45.996496 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 12379
I20260926 15:02:45.996675 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:02:45.997197 9709 external_mini_cluster.cc:1428] Setting key e1c403bdce70b4ba878f4120e2ccef7e
Sep 26 15:02:46 dist-test-slave-7153 krb5kdc[12139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434965, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.123.67@KRBTEST.COM; defaulting to no policy
I20260926 15:02:46.017186 12375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:45.997609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:35705 (local address 127.9.123.126:43511)
0926 15:02:45.997768 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:45.997774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:45.999199 (+ 1425us) server_negotiation.cc:407] Connection header received
0926 15:02:46.001813 (+ 2614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:46.001817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:46.001889 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:46.001978 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:46.004149 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:46.004865 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:46.006572 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:46.007207 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:46.007331 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:46.011228 (+ 3897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:46.011234 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:46.011237 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:46.011280 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:46.013853 (+ 2573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:46.014828 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:46.014831 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:46.014834 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:46.014956 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:46.015451 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:46.015455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:46.015457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:46.015726 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:46.015849 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:46.016652 (+ 803us) server_negotiation.cc:299] Negotiation successful
0926 15:02:46.016847 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.67@KRBTEST.COM" created.
I20260926 15:02:46.018203 12509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:45.998236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:43511 (local address 127.9.123.66:35705)
0926 15:02:45.999054 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:45.999105 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:46.001456 (+ 2351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:46.002300 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:46.002313 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:46.003071 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:46.003913 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:46.003938 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:46.005036 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:46.005042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:46.006399 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:46.006411 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:46.007432 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:46.007438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:46.007644 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:46.008323 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:46.008351 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:46.010970 (+ 2619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:46.014023 (+ 3053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:46.014033 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:46.014054 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:46.014705 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:46.015063 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:46.015068 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:46.015072 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:46.015339 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:46.015835 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:46.015842 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:46.016236 (+ 394us) client_negotiation.cc:769] Sending connection context
0926 15:02:46.016475 (+ 239us) client_negotiation.cc:240] Negotiation successful
0926 15:02:46.016911 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":304,"threads_started":1}
I20260926 15:02:46.019716 12507 heartbeater.cc:347] Connected to a master server at 127.9.123.126:43511
I20260926 15:02:46.020053 12507 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:46.020937 12507 heartbeater.cc:511] Master 127.9.123.126:43511 requested a full tablet report, sending...
I20260926 15:02:46.022610 12184 ts_manager.cc:194] Registered new tserver with Master: eaf3968d07cb43ffae74accb20328a88 (127.9.123.66:39081)
I20260926 15:02:46.023546 12184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.66@KRBTEST.COM'} at 127.9.123.66:35705
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
I20260926 15:02:46.044337 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:43511
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:45431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:46.370064 12512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:46.370352 12512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:46.370465 12512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:46.377838 12512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:46.377965 12512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:46.378115 12512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:02:46.388258 12512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:45431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:43511
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:46.389613 12512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:46.391240 12512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:46.404157 12520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:46.404078 12518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:46.404294 12517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:46.405002 12512 server_base.cc:1034] running on GCE node
I20260926 15:02:46.405669 12512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:46.406638 12512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:46.407837 12512 hybrid_clock.cc:648] HybridClock initialized: now 1790434966407774 us; error 57 us; skew 500 ppm
I20260926 15:02:46.408149 12512 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:46 dist-test-slave-7153 krb5kdc[12139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434966, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:46.411347 12512 init.cc:377] Logged in from keytab as kudu/127.9.123.67@KRBTEST.COM (short username kudu)
I20260926 15:02:46.412820 12512 webserver.cc:460] Webserver started at http://127.9.123.67:44671/ using document root <none> and password file <none>
I20260926 15:02:46.413532 12512 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:46.413646 12512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:46.413944 12512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:46.416868 12512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "789c07fc393f4f9bb07fb5efd6a53f91"
format_stamp: "Formatted at 2026-09-26 15:02:46 on dist-test-slave-7153"
server_key: "84baec2b0318551a13e4b8b4c0cbd13b"
server_key_iv: "b4369ac02f79c34908e28fe2e428c171"
server_key_version: "encryptionkey@0"
I20260926 15:02:46.417723 12512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "789c07fc393f4f9bb07fb5efd6a53f91"
format_stamp: "Formatted at 2026-09-26 15:02:46 on dist-test-slave-7153"
server_key: "84baec2b0318551a13e4b8b4c0cbd13b"
server_key_iv: "b4369ac02f79c34908e28fe2e428c171"
server_key_version: "encryptionkey@0"
I20260926 15:02:46.423138 12512 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 15:02:46.427251 12527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:46.428830 12512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 15:02:46.429003 12512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "789c07fc393f4f9bb07fb5efd6a53f91"
format_stamp: "Formatted at 2026-09-26 15:02:46 on dist-test-slave-7153"
server_key: "84baec2b0318551a13e4b8b4c0cbd13b"
server_key_iv: "b4369ac02f79c34908e28fe2e428c171"
server_key_version: "encryptionkey@0"
I20260926 15:02:46.429255 12512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:46.450577 12512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:46.454907 12512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:46.455181 12512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:46.456212 12512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:02:46.457508 12512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:46.457583 12512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:46.457693 12512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:46.457742 12512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:46.488409 12512 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:39181
I20260926 15:02:46.488454 12639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:39181 every 8 connection(s)
I20260926 15:02:46.489681 12512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:02:46.492309 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 12512
I20260926 15:02:46.492497 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:02:46.493007 9709 external_mini_cluster.cc:1428] Setting key ae90c60129327f3039ce929eeae1fb11
Sep 26 15:02:46 dist-test-slave-7153 krb5kdc[12139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434966, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:46.510010 12375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:46.492133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:60405 (local address 127.9.123.126:43511)
0926 15:02:46.492302 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:46.492306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:46.494426 (+ 2120us) server_negotiation.cc:407] Connection header received
0926 15:02:46.496325 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:46.496331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:46.496437 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:46.496520 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:46.498154 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:46.498957 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:46.500542 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:46.500952 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:46.501067 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:46.504519 (+ 3452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:46.504524 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:46.504527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:46.504568 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:46.507025 (+ 2457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:46.507724 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:46.507728 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:46.507731 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:46.507874 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:46.508283 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:46.508287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:46.508289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:46.508566 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:46.508691 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:46.509451 (+ 760us) server_negotiation.cc:299] Negotiation successful
0926 15:02:46.509673 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 15:02:46.510969 12642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:46.492784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:43511 (local address 127.9.123.67:60405)
0926 15:02:46.494246 (+ 1462us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:46.494306 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:46.496054 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:46.496787 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:46.496796 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:46.497389 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:46.497984 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:46.498001 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:46.499107 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:46.499111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:46.500385 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:46.500398 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:46.501224 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:46.501232 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:46.501461 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:46.502132 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:46.502162 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:46.504294 (+ 2132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:46.507162 (+ 2868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:46.507172 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:46.507190 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:46.507601 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:46.507975 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:46.507980 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:46.507984 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:46.508174 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:46.508686 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:46.508694 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:46.509034 (+ 340us) client_negotiation.cc:769] Sending connection context
0926 15:02:46.509336 (+ 302us) client_negotiation.cc:240] Negotiation successful
0926 15:02:46.509870 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1052,"thread_start_us":804,"threads_started":1}
I20260926 15:02:46.512416 12640 heartbeater.cc:347] Connected to a master server at 127.9.123.126:43511
I20260926 15:02:46.512727 12640 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:46.513501 12640 heartbeater.cc:511] Master 127.9.123.126:43511 requested a full tablet report, sending...
I20260926 15:02:46.515025 12184 ts_manager.cc:194] Registered new tserver with Master: 789c07fc393f4f9bb07fb5efd6a53f91 (127.9.123.67:39181)
I20260926 15:02:46.515867 12184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.67@KRBTEST.COM'} at 127.9.123.67:60405
I20260926 15:02:46.520654 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 15:02:46.534757 12373 heartbeater.cc:503] Master 127.9.123.126:43511 was elected leader, sending a full tablet report...
W20260926 15:02:46.549654 12375 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:02:46.547223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32820 (local address 127.9.123.126:43511)
0926 15:02:46.547426 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:46.547431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:46.547775 (+ 344us) server_negotiation.cc:407] Connection header received
0926 15:02:46.547965 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:46.547969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:46.548038 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:46.548147 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:46.549431 (+ 1284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32820: BlockingRecv error: recv got EOF from 127.0.0.1:32820 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 15:02:46.552548 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 12245
W20260926 15:02:46.555327 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 12245: Not authorized: Client connection negotiation failed: client connection to 127.9.123.65:39677: server requires authentication, but client does not have Kerberos credentials available
I20260926 15:02:46.555475 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 12245
I20260926 15:02:46.567801 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 12379
W20260926 15:02:46.570541 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 12379: Not authorized: Client connection negotiation failed: client connection to 127.9.123.66:39081: server requires authentication, but client does not have Kerberos credentials available
I20260926 15:02:46.570773 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 12379
W20260926 15:02:46.570870 12652 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:02:46.568787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38712 (local address 127.9.123.66:39081)
0926 15:02:46.569365 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:46.569372 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:46.569399 (+ 27us) server_negotiation.cc:407] Connection header received
0926 15:02:46.569459 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:46.569465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:46.569677 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:46.569813 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:46.570516 (+ 703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38712: BlockingRecv error: recv got EOF from 127.0.0.1:38712 (error 108)
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":317,"threads_started":1}
I20260926 15:02:46.582789 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 12512
W20260926 15:02:46.585911 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 12512: Not authorized: Client connection negotiation failed: client connection to 127.9.123.67:39181: server requires authentication, but client does not have Kerberos credentials available
I20260926 15:02:46.586062 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 12512
I20260926 15:02:46.597738 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 12152
I20260926 15:02:46.599007 12214 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:02:46.698056 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 12152
2026-09-26T15:02:46Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3405 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:02:46 dist-test-slave-7153 krb5kdc[12657](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:02:46 dist-test-slave-7153 krb5kdc[12657](info): set up 2 sockets
Sep 26 15:02:46 dist-test-slave-7153 krb5kdc[12657](info): commencing operation
krb5kdc: starting...
W20260926 15:02:47.705322 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.970s user 0.004s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:02:47 dist-test-slave-7153 krb5kdc[12657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:02:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:02:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:02:47.892177 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:34381
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:42553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:34381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:48.220633 12670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:48.220929 12670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:48.221042 12670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:48.228398 12670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:02:48.228518 12670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:48.228587 12670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:48.228667 12670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:02:48.228734 12670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:02:48.238776 12670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:42553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:34381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:34381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:48.240059 12670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:48.241393 12670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:48.252939 12675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:48.252887 12676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:48.252858 12678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:48.253762 12670 server_base.cc:1034] running on GCE node
I20260926 15:02:48.254575 12670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:48.256043 12670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:48.257203 12670 hybrid_clock.cc:648] HybridClock initialized: now 1790434968257143 us; error 47 us; skew 500 ppm
I20260926 15:02:48.257545 12670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:48 dist-test-slave-7153 krb5kdc[12657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434968, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:48.260547 12670 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:02:48.262107 12670 webserver.cc:460] Webserver started at http://127.9.123.126:34335/ using document root <none> and password file <none>
I20260926 15:02:48.262789 12670 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:48.262907 12670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:48.263197 12670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:48.266259 12670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "332a077e29e240588deec0e53a91d95a"
format_stamp: "Formatted at 2026-09-26 15:02:48 on dist-test-slave-7153"
server_key: "d02046bd684513353e99d1e5814f8f15"
server_key_iv: "125ab2600401710e74ab5fe502aecba6"
server_key_version: "encryptionkey@0"
I20260926 15:02:48.267069 12670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "332a077e29e240588deec0e53a91d95a"
format_stamp: "Formatted at 2026-09-26 15:02:48 on dist-test-slave-7153"
server_key: "d02046bd684513353e99d1e5814f8f15"
server_key_iv: "125ab2600401710e74ab5fe502aecba6"
server_key_version: "encryptionkey@0"
I20260926 15:02:48.272316 12670 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 15:02:48.275718 12685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:48.277022 12670 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 15:02:48.277221 12670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "332a077e29e240588deec0e53a91d95a"
format_stamp: "Formatted at 2026-09-26 15:02:48 on dist-test-slave-7153"
server_key: "d02046bd684513353e99d1e5814f8f15"
server_key_iv: "125ab2600401710e74ab5fe502aecba6"
server_key_version: "encryptionkey@0"
I20260926 15:02:48.277400 12670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:48.291591 12670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:48.295697 12670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:48.295953 12670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:48.312947 12670 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:34381
I20260926 15:02:48.312978 12736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:34381 every 8 connection(s)
I20260926 15:02:48.314446 12670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:02:48.318253 12737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:48.319873 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 12670
I20260926 15:02:48.320092 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:02:48.320856 9709 external_mini_cluster.cc:1428] Setting key fa0a6c97426f391f14b3fbcfab65a53f
I20260926 15:02:48.333081 12737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 332a077e29e240588deec0e53a91d95a: Bootstrap starting.
Sep 26 15:02:48 dist-test-slave-7153 krb5kdc[12657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:48.336905 12737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 332a077e29e240588deec0e53a91d95a: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:48.337988 12737 log.cc:824] T 00000000000000000000000000000000 P 332a077e29e240588deec0e53a91d95a: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:48.341130 12737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 332a077e29e240588deec0e53a91d95a: No bootstrap required, opened a new log
I20260926 15:02:48.344036 12740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:48.322870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51086 (local address 127.9.123.126:34381)
0926 15:02:48.323746 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:48.323758 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:48.323801 (+ 43us) server_negotiation.cc:407] Connection header received
0926 15:02:48.325391 (+ 1590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:48.325421 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:48.325875 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:48.326228 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:48.327935 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:48.329321 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:48.331066 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:48.331769 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:48.331898 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:48.336425 (+ 4527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:48.336434 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:48.336450 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:48.336489 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:48.339401 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:48.340273 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:48.340280 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:48.340287 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:48.340461 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:48.340905 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:48.340909 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:48.340912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:48.341565 (+ 653us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:48.341841 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:48.342406 (+ 565us) server_negotiation.cc:299] Negotiation successful
0926 15:02:48.342883 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":413,"threads_started":1}
I20260926 15:02:48.348101 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:02:48.349857 12737 raft_consensus.cc:348] T 00000000000000000000000000000000 P 332a077e29e240588deec0e53a91d95a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "332a077e29e240588deec0e53a91d95a" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 34381 } }
I20260926 15:02:48.350333 12737 raft_consensus.cc:374] T 00000000000000000000000000000000 P 332a077e29e240588deec0e53a91d95a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:48.350442 12737 raft_consensus.cc:729] T 00000000000000000000000000000000 P 332a077e29e240588deec0e53a91d95a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 332a077e29e240588deec0e53a91d95a, State: Initialized, Role: FOLLOWER
I20260926 15:02:48.351145 12737 consensus_queue.cc:260] T 00000000000000000000000000000000 P 332a077e29e240588deec0e53a91d95a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "332a077e29e240588deec0e53a91d95a" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 34381 } }
I20260926 15:02:48.351424 12737 raft_consensus.cc:388] T 00000000000000000000000000000000 P 332a077e29e240588deec0e53a91d95a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:02:48.351553 12737 raft_consensus.cc:482] T 00000000000000000000000000000000 P 332a077e29e240588deec0e53a91d95a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:02:48.351718 12737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 332a077e29e240588deec0e53a91d95a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:48.354736 12737 raft_consensus.cc:504] T 00000000000000000000000000000000 P 332a077e29e240588deec0e53a91d95a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "332a077e29e240588deec0e53a91d95a" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 34381 } }
I20260926 15:02:48.355350 12737 leader_election.cc:302] T 00000000000000000000000000000000 P 332a077e29e240588deec0e53a91d95a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 332a077e29e240588deec0e53a91d95a; no voters:
I20260926 15:02:48.356158 12737 leader_election.cc:288] T 00000000000000000000000000000000 P 332a077e29e240588deec0e53a91d95a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:02:48.356277 12743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 332a077e29e240588deec0e53a91d95a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:48.357409 12743 raft_consensus.cc:686] T 00000000000000000000000000000000 P 332a077e29e240588deec0e53a91d95a [term 1 LEADER]: Becoming Leader. State: Replica: 332a077e29e240588deec0e53a91d95a, State: Running, Role: LEADER
I20260926 15:02:48.358045 12743 consensus_queue.cc:237] T 00000000000000000000000000000000 P 332a077e29e240588deec0e53a91d95a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "332a077e29e240588deec0e53a91d95a" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 34381 } }
I20260926 15:02:48.358606 12737 sys_catalog.cc:564] T 00000000000000000000000000000000 P 332a077e29e240588deec0e53a91d95a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:02:48.361195 12745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 332a077e29e240588deec0e53a91d95a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 332a077e29e240588deec0e53a91d95a. Latest consensus state: current_term: 1 leader_uuid: "332a077e29e240588deec0e53a91d95a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "332a077e29e240588deec0e53a91d95a" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 34381 } } }
I20260926 15:02:48.361603 12745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 332a077e29e240588deec0e53a91d95a [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:48.362178 12744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 332a077e29e240588deec0e53a91d95a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "332a077e29e240588deec0e53a91d95a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "332a077e29e240588deec0e53a91d95a" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 34381 } } }
I20260926 15:02:48.362473 12744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 332a077e29e240588deec0e53a91d95a [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:48.374177 12749 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:02:48.379655 12749 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:02:48.403162 12749 catalog_manager.cc:1347] Generated new cluster ID: 1a735268d1ec4ea0a5656cac7dad6381
I20260926 15:02:48.403322 12749 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:02:48.440448 12749 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:02:48.443005 12749 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:02:48.446920 12740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:48.426883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51096 (local address 127.9.123.126:34381)
0926 15:02:48.427073 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:48.427082 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:48.436803 (+ 9721us) server_negotiation.cc:407] Connection header received
0926 15:02:48.437377 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:48.437384 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:48.437479 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:48.437558 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:48.439063 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:48.440195 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:48.442584 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:48.443289 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:48.443422 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:48.445076 (+ 1654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:48.445084 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:48.445089 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:02:48.445956 (+ 867us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:48.446041 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:02:48.446071 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:48.446177 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:48.446393 (+ 216us) server_negotiation.cc:299] Negotiation successful
0926 15:02:48.446534 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 15:02:48.459605 12749 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 332a077e29e240588deec0e53a91d95a: Generated new TSK 0
I20260926 15:02:48.460502 12749 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:02:48.467461 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 12670
I20260926 15:02:48.469172 12732 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:02:48.568485 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 12670
2026-09-26T15:02:48Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1866 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:02:48 dist-test-slave-7153 krb5kdc[12771](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:02:48 dist-test-slave-7153 krb5kdc[12771](info): set up 2 sockets
Sep 26 15:02:48 dist-test-slave-7153 krb5kdc[12771](info): commencing operation
krb5kdc: starting...
W20260926 15:02:49.565415 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.964s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:02:49 dist-test-slave-7153 krb5kdc[12771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:02:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:02:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:02:49.748483 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:38029
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:43333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:38029
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:50.070188 12784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:50.070495 12784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:50.070609 12784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:50.078045 12784 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:02:50.078179 12784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:50.078234 12784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:50.078313 12784 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:02:50.078370 12784 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:02:50.089898 12784 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:43333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:38029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:38029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:50.091301 12784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:50.092649 12784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:50.104590 12792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:50.104620 12790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:50.104657 12789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:50.105381 12784 server_base.cc:1034] running on GCE node
I20260926 15:02:50.106211 12784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:50.107697 12784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:50.108923 12784 hybrid_clock.cc:648] HybridClock initialized: now 1790434970108864 us; error 44 us; skew 500 ppm
I20260926 15:02:50.109338 12784 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:50 dist-test-slave-7153 krb5kdc[12771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434970, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:50.112679 12784 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:02:50.114264 12784 webserver.cc:460] Webserver started at http://127.9.123.126:40351/ using document root <none> and password file <none>
I20260926 15:02:50.114943 12784 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:50.115068 12784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:50.115399 12784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:50.118435 12784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "4aca64b935d5436f96ba7dc80a1e6155"
format_stamp: "Formatted at 2026-09-26 15:02:50 on dist-test-slave-7153"
server_key: "1159d902891393037eb7852367e15183"
server_key_iv: "2f6f5a7b4f572fc271879e57ac12e14a"
server_key_version: "encryptionkey@0"
I20260926 15:02:50.119187 12784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "4aca64b935d5436f96ba7dc80a1e6155"
format_stamp: "Formatted at 2026-09-26 15:02:50 on dist-test-slave-7153"
server_key: "1159d902891393037eb7852367e15183"
server_key_iv: "2f6f5a7b4f572fc271879e57ac12e14a"
server_key_version: "encryptionkey@0"
I20260926 15:02:50.124837 12784 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260926 15:02:50.128793 12799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:50.130263 12784 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:02:50.130443 12784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "4aca64b935d5436f96ba7dc80a1e6155"
format_stamp: "Formatted at 2026-09-26 15:02:50 on dist-test-slave-7153"
server_key: "1159d902891393037eb7852367e15183"
server_key_iv: "2f6f5a7b4f572fc271879e57ac12e14a"
server_key_version: "encryptionkey@0"
I20260926 15:02:50.130630 12784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:50.147965 12784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:50.151893 12784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:50.152148 12784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:50.169305 12784 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:38029
I20260926 15:02:50.169325 12850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:38029 every 8 connection(s)
I20260926 15:02:50.170621 12784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:02:50.174233 12851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:50.176561 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 12784
I20260926 15:02:50.176812 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:02:50.177597 9709 external_mini_cluster.cc:1428] Setting key 3b73f328a339b929549daf094dcb7ba9
I20260926 15:02:50.187038 12851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4aca64b935d5436f96ba7dc80a1e6155: Bootstrap starting.
I20260926 15:02:50.190864 12851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4aca64b935d5436f96ba7dc80a1e6155: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:50.192130 12851 log.cc:824] T 00000000000000000000000000000000 P 4aca64b935d5436f96ba7dc80a1e6155: Log is configured to *not* fsync() on all Append() calls
Sep 26 15:02:50 dist-test-slave-7153 krb5kdc[12771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:50.195816 12851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4aca64b935d5436f96ba7dc80a1e6155: No bootstrap required, opened a new log
I20260926 15:02:50.202981 12854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:50.179828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41648 (local address 127.9.123.126:38029)
0926 15:02:50.180615 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:50.180628 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:50.180671 (+ 43us) server_negotiation.cc:407] Connection header received
0926 15:02:50.182420 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:50.182447 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:50.182893 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:50.183274 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:50.184967 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:50.186359 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:50.188072 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:50.188776 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:50.188922 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:50.193684 (+ 4762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:50.193698 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:50.193719 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:50.193774 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:50.197734 (+ 3960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:50.198607 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:50.198628 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:50.198638 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:50.198855 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:50.199321 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:50.199331 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:50.199335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:50.200114 (+ 779us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:50.200445 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:50.200925 (+ 480us) server_negotiation.cc:299] Negotiation successful
0926 15:02:50.201477 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":339,"threads_started":1}
I20260926 15:02:50.205684 12851 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4aca64b935d5436f96ba7dc80a1e6155 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aca64b935d5436f96ba7dc80a1e6155" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 38029 } }
I20260926 15:02:50.206060 12851 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4aca64b935d5436f96ba7dc80a1e6155 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:50.206161 12851 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4aca64b935d5436f96ba7dc80a1e6155 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aca64b935d5436f96ba7dc80a1e6155, State: Initialized, Role: FOLLOWER
I20260926 15:02:50.206702 12851 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4aca64b935d5436f96ba7dc80a1e6155 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aca64b935d5436f96ba7dc80a1e6155" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 38029 } }
I20260926 15:02:50.206920 12851 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4aca64b935d5436f96ba7dc80a1e6155 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:02:50.207041 12851 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4aca64b935d5436f96ba7dc80a1e6155 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:02:50.207183 12851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4aca64b935d5436f96ba7dc80a1e6155 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:50.209311 12851 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4aca64b935d5436f96ba7dc80a1e6155 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aca64b935d5436f96ba7dc80a1e6155" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 38029 } }
I20260926 15:02:50.209795 12851 leader_election.cc:302] T 00000000000000000000000000000000 P 4aca64b935d5436f96ba7dc80a1e6155 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4aca64b935d5436f96ba7dc80a1e6155; no voters:
I20260926 15:02:50.210353 12851 leader_election.cc:288] T 00000000000000000000000000000000 P 4aca64b935d5436f96ba7dc80a1e6155 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:02:50.210471 12856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4aca64b935d5436f96ba7dc80a1e6155 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:50.211123 12856 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4aca64b935d5436f96ba7dc80a1e6155 [term 1 LEADER]: Becoming Leader. State: Replica: 4aca64b935d5436f96ba7dc80a1e6155, State: Running, Role: LEADER
I20260926 15:02:50.211674 12856 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4aca64b935d5436f96ba7dc80a1e6155 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aca64b935d5436f96ba7dc80a1e6155" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 38029 } }
I20260926 15:02:50.212344 12851 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4aca64b935d5436f96ba7dc80a1e6155 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:02:50.214340 12857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4aca64b935d5436f96ba7dc80a1e6155 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4aca64b935d5436f96ba7dc80a1e6155" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aca64b935d5436f96ba7dc80a1e6155" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 38029 } } }
I20260926 15:02:50.214671 12857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4aca64b935d5436f96ba7dc80a1e6155 [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:50.215626 12861 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:02:50.216465 12858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4aca64b935d5436f96ba7dc80a1e6155 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4aca64b935d5436f96ba7dc80a1e6155. Latest consensus state: current_term: 1 leader_uuid: "4aca64b935d5436f96ba7dc80a1e6155" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aca64b935d5436f96ba7dc80a1e6155" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 38029 } } }
I20260926 15:02:50.216801 12858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4aca64b935d5436f96ba7dc80a1e6155 [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:50.221926 12861 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:02:50.230935 12861 catalog_manager.cc:1347] Generated new cluster ID: 576c812cf0044de9b0c0d580bcf9ffeb
I20260926 15:02:50.231063 12861 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:02:50.254161 12861 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:02:50.255553 12861 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:02:50.261135 12861 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4aca64b935d5436f96ba7dc80a1e6155: Generated new TSK 0
I20260926 15:02:50.262205 12861 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.123.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
I20260926 15:02:50.326967 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:38029
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:43333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:50.661787 12877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:50.662084 12877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:50.662216 12877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:50.669443 12877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:50.669561 12877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:50.669706 12877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:02:50.679481 12877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:43333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:38029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:50.680786 12877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:50.682127 12877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:50.694804 12883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:50.694847 12885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:50.694878 12882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:50.695885 12877 server_base.cc:1034] running on GCE node
I20260926 15:02:50.696529 12877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:50.697525 12877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:50.698730 12877 hybrid_clock.cc:648] HybridClock initialized: now 1790434970698675 us; error 51 us; skew 500 ppm
I20260926 15:02:50.699045 12877 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:50 dist-test-slave-7153 krb5kdc[12771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434970, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:50.702314 12877 init.cc:377] Logged in from keytab as kudu/127.9.123.65@KRBTEST.COM (short username kudu)
I20260926 15:02:50.703814 12877 webserver.cc:460] Webserver started at http://127.9.123.65:33895/ using document root <none> and password file <none>
I20260926 15:02:50.704511 12877 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:50.704631 12877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:50.704924 12877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:50.707902 12877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "50999e598b4e45a2b3c351ac02db27f1"
format_stamp: "Formatted at 2026-09-26 15:02:50 on dist-test-slave-7153"
server_key: "0d109f34058504ea67f2d4ed492058e2"
server_key_iv: "ff19410e34b3585575c5cffa3bcb53e8"
server_key_version: "encryptionkey@0"
I20260926 15:02:50.708666 12877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "50999e598b4e45a2b3c351ac02db27f1"
format_stamp: "Formatted at 2026-09-26 15:02:50 on dist-test-slave-7153"
server_key: "0d109f34058504ea67f2d4ed492058e2"
server_key_iv: "ff19410e34b3585575c5cffa3bcb53e8"
server_key_version: "encryptionkey@0"
I20260926 15:02:50.714035 12877 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 15:02:50.717790 12892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:50.719463 12877 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:02:50.719637 12877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "50999e598b4e45a2b3c351ac02db27f1"
format_stamp: "Formatted at 2026-09-26 15:02:50 on dist-test-slave-7153"
server_key: "0d109f34058504ea67f2d4ed492058e2"
server_key_iv: "ff19410e34b3585575c5cffa3bcb53e8"
server_key_version: "encryptionkey@0"
I20260926 15:02:50.719834 12877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:50.750931 12877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:50.756023 12877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:50.756349 12877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:50.757428 12877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:02:50.758901 12877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:50.758985 12877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:50.759097 12877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:50.759148 12877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:50.792536 12877 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:37089
I20260926 15:02:50.792587 13004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:37089 every 8 connection(s)
I20260926 15:02:50.793879 12877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:02:50.798430 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 12877
I20260926 15:02:50.798610 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:02:50.799207 9709 external_mini_cluster.cc:1428] Setting key 273ab51e2faf2ec04dd8fec7630a72c8
Sep 26 15:02:50 dist-test-slave-7153 krb5kdc[12771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434970, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:50.814277 13007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:50.796453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:35105 (local address 127.9.123.126:38029)
0926 15:02:50.796954 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:50.796959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:50.797938 (+ 979us) server_negotiation.cc:407] Connection header received
0926 15:02:50.799927 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:50.799933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:50.800031 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:50.800126 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:50.801809 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:50.802489 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:50.804255 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:50.804669 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:50.804789 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:50.808529 (+ 3740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:50.808535 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:50.808538 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:50.808574 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:50.811098 (+ 2524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:50.811889 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:50.811894 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:50.811896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:50.812022 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:50.812486 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:50.812492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:50.812495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:50.812755 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:50.812884 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:50.813696 (+ 812us) server_negotiation.cc:299] Negotiation successful
0926 15:02:50.813927 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":293,"threads_started":1}
WARNING: no policy specified for kudu/127.9.123.66@KRBTEST.COM; defaulting to no policy
I20260926 15:02:50.815313 13008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:50.796998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:38029 (local address 127.9.123.65:35105)
0926 15:02:50.797757 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:50.797809 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:50.799617 (+ 1808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:50.800407 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:50.800418 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:50.800954 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:50.801632 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:50.801651 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:50.802656 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:50.802660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:50.803966 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:50.803980 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:50.804914 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:50.804922 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:50.805218 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:50.806005 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:50.806033 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:50.808274 (+ 2241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:50.811276 (+ 3002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:50.811286 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:50.811304 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:50.811749 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:50.812146 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:50.812149 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:50.812151 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:50.812349 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:50.812889 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:50.812898 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:50.813307 (+ 409us) client_negotiation.cc:769] Sending connection context
0926 15:02:50.813579 (+ 272us) client_negotiation.cc:240] Negotiation successful
0926 15:02:50.814150 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":283,"threads_started":1}
I20260926 15:02:50.817025 13005 heartbeater.cc:347] Connected to a master server at 127.9.123.126:38029
I20260926 15:02:50.817380 13005 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.66@KRBTEST.COM" created.
I20260926 15:02:50.818497 13005 heartbeater.cc:511] Master 127.9.123.126:38029 requested a full tablet report, sending...
I20260926 15:02:50.820827 12816 ts_manager.cc:194] Registered new tserver with Master: 50999e598b4e45a2b3c351ac02db27f1 (127.9.123.65:37089)
I20260926 15:02:50.822669 12816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.65@KRBTEST.COM'} at 127.9.123.65:35105
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
I20260926 15:02:50.846277 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:38029
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:43333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:51.175767 13011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:51.176128 13011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:51.176263 13011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:51.187713 13011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:51.187860 13011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:51.188035 13011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:02:51.199226 13011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:43333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:38029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:51.200560 13011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:51.201915 13011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:51.215538 13019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:51.215633 13017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:51.215632 13016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:51.216117 13011 server_base.cc:1034] running on GCE node
I20260926 15:02:51.216881 13011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:51.217980 13011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:51.219198 13011 hybrid_clock.cc:648] HybridClock initialized: now 1790434971219151 us; error 43 us; skew 500 ppm
I20260926 15:02:51.219547 13011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:51 dist-test-slave-7153 krb5kdc[12771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434971, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:51.227099 13011 init.cc:377] Logged in from keytab as kudu/127.9.123.66@KRBTEST.COM (short username kudu)
I20260926 15:02:51.228677 13011 webserver.cc:460] Webserver started at http://127.9.123.66:44529/ using document root <none> and password file <none>
I20260926 15:02:51.229485 13011 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:51.229621 13011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:51.229924 13011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:51.232932 13011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "db03df5025524f1c89cda7e92dc854d5"
format_stamp: "Formatted at 2026-09-26 15:02:51 on dist-test-slave-7153"
server_key: "2965c1c6d9eaea94d95c8a5db1466867"
server_key_iv: "2a40b2b9a895300b18328fa50de6480c"
server_key_version: "encryptionkey@0"
I20260926 15:02:51.233845 13011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "db03df5025524f1c89cda7e92dc854d5"
format_stamp: "Formatted at 2026-09-26 15:02:51 on dist-test-slave-7153"
server_key: "2965c1c6d9eaea94d95c8a5db1466867"
server_key_iv: "2a40b2b9a895300b18328fa50de6480c"
server_key_version: "encryptionkey@0"
I20260926 15:02:51.239259 13011 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260926 15:02:51.242873 13026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:51.244347 13011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 15:02:51.244510 13011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "db03df5025524f1c89cda7e92dc854d5"
format_stamp: "Formatted at 2026-09-26 15:02:51 on dist-test-slave-7153"
server_key: "2965c1c6d9eaea94d95c8a5db1466867"
server_key_iv: "2a40b2b9a895300b18328fa50de6480c"
server_key_version: "encryptionkey@0"
I20260926 15:02:51.244695 13011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:51.258419 13011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:51.262990 13011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:51.263288 13011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:51.264304 13011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:02:51.265755 13011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:51.265833 13011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:51.265954 13011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:51.266005 13011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:51.295845 13011 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:37409
I20260926 15:02:51.295933 13138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:37409 every 8 connection(s)
I20260926 15:02:51.297140 13011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:02:51.305171 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 13011
I20260926 15:02:51.305346 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:02:51.305763 9709 external_mini_cluster.cc:1428] Setting key 034febecf3c0c0bef376a0779b6c424d
Sep 26 15:02:51 dist-test-slave-7153 krb5kdc[12771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434971, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:51.318320 13007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:51.299814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:58629 (local address 127.9.123.126:38029)
0926 15:02:51.300024 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:51.300030 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:51.301226 (+ 1196us) server_negotiation.cc:407] Connection header received
0926 15:02:51.303021 (+ 1795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:51.303026 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:51.303107 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:51.303190 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:51.304815 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:51.305617 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:51.307720 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:51.308314 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:51.308459 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:51.312104 (+ 3645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:51.312111 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:51.312115 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:51.312155 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:51.314889 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:51.315810 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:51.315820 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:51.315823 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:51.315990 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:51.316472 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:51.316476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:51.316478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:51.316787 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:51.316898 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:51.317822 (+ 924us) server_negotiation.cc:299] Negotiation successful
0926 15:02:51.318009 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 15:02:51.319725 13141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:51.300313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:38029 (local address 127.9.123.66:58629)
0926 15:02:51.301042 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:51.301087 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:51.302744 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:51.303441 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:51.303449 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:51.303966 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:51.304560 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:51.304578 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:51.305771 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:51.305776 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:51.307497 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:51.307520 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:51.308563 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:51.308569 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:51.308791 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:51.309518 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:51.309542 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:51.311849 (+ 2307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:51.315041 (+ 3192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:51.315050 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:51.315068 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:51.315676 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:51.316110 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:51.316114 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:51.316117 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:51.316358 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:51.316907 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:51.316916 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:51.317381 (+ 465us) client_negotiation.cc:769] Sending connection context
0926 15:02:51.317701 (+ 320us) client_negotiation.cc:240] Negotiation successful
0926 15:02:51.318282 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":269,"threads_started":1}
I20260926 15:02:51.321337 13139 heartbeater.cc:347] Connected to a master server at 127.9.123.126:38029
I20260926 15:02:51.321640 13139 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:51.322394 13139 heartbeater.cc:511] Master 127.9.123.126:38029 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.123.67@KRBTEST.COM; defaulting to no policy
I20260926 15:02:51.323858 12816 ts_manager.cc:194] Registered new tserver with Master: db03df5025524f1c89cda7e92dc854d5 (127.9.123.66:37409)
I20260926 15:02:51.324693 12816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.66@KRBTEST.COM'} at 127.9.123.66:58629
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
I20260926 15:02:51.348376 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:38029
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:43333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:51.667550 13144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:51.667869 13144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:51.667995 13144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:51.675316 13144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:51.675454 13144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:51.675596 13144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:02:51.685369 13144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:43333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:38029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:51.686702 13144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:51.688040 13144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:51.700881 13149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:51.700914 13152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:51.700944 13150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:51.702165 13144 server_base.cc:1034] running on GCE node
I20260926 15:02:51.702867 13144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:51.703910 13144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:51.705173 13144 hybrid_clock.cc:648] HybridClock initialized: now 1790434971705115 us; error 52 us; skew 500 ppm
I20260926 15:02:51.705507 13144 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:51 dist-test-slave-7153 krb5kdc[12771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434971, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:51.708576 13144 init.cc:377] Logged in from keytab as kudu/127.9.123.67@KRBTEST.COM (short username kudu)
I20260926 15:02:51.710206 13144 webserver.cc:460] Webserver started at http://127.9.123.67:46287/ using document root <none> and password file <none>
I20260926 15:02:51.710945 13144 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:51.711066 13144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:51.711398 13144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:51.714521 13144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "86fb60d4f64c418cb2e6f4b832998859"
format_stamp: "Formatted at 2026-09-26 15:02:51 on dist-test-slave-7153"
server_key: "99adbe4943aa0ce8ccb4683879bb13f1"
server_key_iv: "fb6dc6dcad3769a49748bff30a3ca0da"
server_key_version: "encryptionkey@0"
I20260926 15:02:51.715350 13144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "86fb60d4f64c418cb2e6f4b832998859"
format_stamp: "Formatted at 2026-09-26 15:02:51 on dist-test-slave-7153"
server_key: "99adbe4943aa0ce8ccb4683879bb13f1"
server_key_iv: "fb6dc6dcad3769a49748bff30a3ca0da"
server_key_version: "encryptionkey@0"
I20260926 15:02:51.720861 13144 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 15:02:51.724673 13159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:51.726235 13144 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:02:51.726418 13144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "86fb60d4f64c418cb2e6f4b832998859"
format_stamp: "Formatted at 2026-09-26 15:02:51 on dist-test-slave-7153"
server_key: "99adbe4943aa0ce8ccb4683879bb13f1"
server_key_iv: "fb6dc6dcad3769a49748bff30a3ca0da"
server_key_version: "encryptionkey@0"
I20260926 15:02:51.726629 13144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:51.742334 13144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:51.746848 13144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:51.747159 13144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:51.748461 13144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:02:51.749888 13144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:51.749969 13144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:51.750087 13144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:51.750149 13144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:51.780967 13144 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:36945
I20260926 15:02:51.781019 13271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:36945 every 8 connection(s)
I20260926 15:02:51.782279 13144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:02:51.785974 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 13144
I20260926 15:02:51.786134 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:02:51.786623 9709 external_mini_cluster.cc:1428] Setting key b3879463698026c2e69e4212539139db
Sep 26 15:02:51 dist-test-slave-7153 krb5kdc[12771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434971, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:51.802021 13007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:51.784880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:47751 (local address 127.9.123.126:38029)
0926 15:02:51.785044 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:51.785049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:51.786413 (+ 1364us) server_negotiation.cc:407] Connection header received
0926 15:02:51.788540 (+ 2127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:51.788544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:51.788636 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:51.788727 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:51.790446 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:51.791201 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:51.792794 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:51.793241 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:51.793351 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:51.796726 (+ 3375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:51.796732 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:51.796735 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:51.796768 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:51.799043 (+ 2275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:51.799744 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:51.799749 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:51.799751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:51.799876 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:51.800274 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:51.800279 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:51.800282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:51.800549 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:51.800676 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:51.801495 (+ 819us) server_negotiation.cc:299] Negotiation successful
0926 15:02:51.801715 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 15:02:51.803171 13274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:51.785397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:38029 (local address 127.9.123.67:47751)
0926 15:02:51.786224 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:51.786279 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:51.788241 (+ 1962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:51.788962 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:51.788972 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:51.789727 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:51.790304 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:51.790321 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:51.791329 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:51.791334 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:51.792621 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:51.792633 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:51.793452 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:51.793459 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:51.793683 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:51.794348 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:51.794371 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:51.796502 (+ 2131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:51.799188 (+ 2686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:51.799198 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:51.799216 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:51.799622 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:51.799979 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:51.799984 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:51.799988 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:51.800170 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:51.800669 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:51.800677 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:51.801079 (+ 402us) client_negotiation.cc:769] Sending connection context
0926 15:02:51.801415 (+ 336us) client_negotiation.cc:240] Negotiation successful
0926 15:02:51.801929 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":355,"threads_started":1}
I20260926 15:02:51.804587 13272 heartbeater.cc:347] Connected to a master server at 127.9.123.126:38029
I20260926 15:02:51.804880 13272 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:51.805724 13272 heartbeater.cc:511] Master 127.9.123.126:38029 requested a full tablet report, sending...
I20260926 15:02:51.807202 12816 ts_manager.cc:194] Registered new tserver with Master: 86fb60d4f64c418cb2e6f4b832998859 (127.9.123.67:36945)
I20260926 15:02:51.808005 12816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.67@KRBTEST.COM'} at 127.9.123.67:47751
I20260926 15:02:51.815055 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 15:02:51.827047 13005 heartbeater.cc:503] Master 127.9.123.126:38029 was elected leader, sending a full tablet report...
Sep 26 15:02:51 dist-test-slave-7153 krb5kdc[12771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434971, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 26 15:02:51 dist-test-slave-7153 krb5kdc[12771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434971, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:51.861169 13007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:51.847618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41650 (local address 127.9.123.126:38029)
0926 15:02:51.847801 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:51.847807 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:51.848205 (+ 398us) server_negotiation.cc:407] Connection header received
0926 15:02:51.848360 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:51.848364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:51.848426 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:51.848511 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:51.849918 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:51.850638 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:51.851977 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:51.852386 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:51.852485 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:51.856282 (+ 3797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:51.856287 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:51.856290 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:51.856326 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:51.858771 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:51.859367 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:51.859371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:51.859374 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:51.859507 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:51.859919 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:51.859922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:51.859924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:51.860259 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:51.860430 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:51.860705 (+ 275us) server_negotiation.cc:299] Negotiation successful
0926 15:02:51.860875 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260926 15:02:51.861770 12816 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41650
I20260926 15:02:51.863478 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 12877
Sep 26 15:02:51 dist-test-slave-7153 krb5kdc[12771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434971, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.123.65@KRBTEST.COM
I20260926 15:02:51.878571 13283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:51.864513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37516 (local address 127.9.123.65:37089)
0926 15:02:51.865121 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:51.865131 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:51.865160 (+ 29us) server_negotiation.cc:407] Connection header received
0926 15:02:51.865218 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:51.865224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:51.865454 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:51.865576 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:51.866695 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:51.867710 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:51.868865 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:51.869495 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:51.869593 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:51.873400 (+ 3807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:51.873413 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:51.873428 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:51.873469 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:51.876164 (+ 2695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:51.876829 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:51.876836 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:51.876838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:51.876959 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:51.877340 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:51.877343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:51.877345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:51.877686 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:51.877915 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:51.878103 (+ 188us) server_negotiation.cc:299] Negotiation successful
0926 15:02:51.878295 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":303,"threads_started":1}
W20260926 15:02:51.879819 13000 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37516
W20260926 15:02:51.880939 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 12877: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 15:02:51.881157 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 12877
I20260926 15:02:51.893466 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 13011
Sep 26 15:02:51 dist-test-slave-7153 krb5kdc[12771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434971, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.123.66@KRBTEST.COM
I20260926 15:02:51.909307 13284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:51.894340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34604 (local address 127.9.123.66:37409)
0926 15:02:51.894896 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:51.894903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:51.894927 (+ 24us) server_negotiation.cc:407] Connection header received
0926 15:02:51.894984 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:51.894990 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:51.895192 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:51.895320 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:51.896541 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:51.897665 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:51.898927 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:51.899473 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:51.899565 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:51.903702 (+ 4137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:51.903716 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:51.903733 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:51.903776 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:51.906423 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:51.907141 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:51.907149 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:51.907151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:51.907288 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:51.907643 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:51.907647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:51.907649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:51.907993 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:51.908321 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:51.908600 (+ 279us) server_negotiation.cc:299] Negotiation successful
0926 15:02:51.908856 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":288,"threads_started":1}
W20260926 15:02:51.910606 13134 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34604
W20260926 15:02:51.911698 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 13011: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 15:02:51.911851 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 13011
I20260926 15:02:51.923957 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 13144
Sep 26 15:02:51 dist-test-slave-7153 krb5kdc[12771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434971, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.123.67@KRBTEST.COM
I20260926 15:02:51.939467 13285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:51.925011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45154 (local address 127.9.123.67:36945)
0926 15:02:51.925679 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:51.925686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:51.925716 (+ 30us) server_negotiation.cc:407] Connection header received
0926 15:02:51.925779 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:51.925786 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:51.925993 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:51.926113 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:51.927372 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:51.928399 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:51.929600 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:51.930200 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:51.930323 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:51.934437 (+ 4114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:51.934448 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:51.934464 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:51.934500 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:51.937085 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:51.937792 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:51.937800 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:51.937802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:51.937922 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:51.938195 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:51.938198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:51.938200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:51.938515 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:51.938731 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:51.938980 (+ 249us) server_negotiation.cc:299] Negotiation successful
0926 15:02:51.939183 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":374,"threads_started":1}
W20260926 15:02:51.940773 13267 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45154
W20260926 15:02:51.941963 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 13144: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 15:02:51.942112 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 13144
I20260926 15:02:51.954175 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 12784
I20260926 15:02:51.955305 12846 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:02:52.055778 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 12784
2026-09-26T15:02:52Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3492 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:02:52 dist-test-slave-7153 krb5kdc[13289](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:02:52 dist-test-slave-7153 krb5kdc[13289](info): set up 2 sockets
Sep 26 15:02:52 dist-test-slave-7153 krb5kdc[13289](info): commencing operation
krb5kdc: starting...
W20260926 15:02:53.057381 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.964s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:02:53 dist-test-slave-7153 krb5kdc[13289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:02:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:02:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:02:53.248937 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:44329
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:44329
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:53.575708 13302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:53.576026 13302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:53.576114 13302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:53.583293 13302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:02:53.583413 13302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:53.583459 13302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:53.583513 13302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:02:53.583556 13302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:02:53.593652 13302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:44329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:44329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:53.594998 13302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:53.596280 13302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:53.608532 13308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:53.608561 13307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:53.608503 13310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:53.609877 13302 server_base.cc:1034] running on GCE node
I20260926 15:02:53.610703 13302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:53.612205 13302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:53.613404 13302 hybrid_clock.cc:648] HybridClock initialized: now 1790434973613337 us; error 59 us; skew 500 ppm
I20260926 15:02:53.613759 13302 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:53 dist-test-slave-7153 krb5kdc[13289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434973, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:53.617370 13302 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:02:53.619241 13302 webserver.cc:460] Webserver started at http://127.9.123.126:36627/ using document root <none> and password file <none>
I20260926 15:02:53.620007 13302 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:53.620136 13302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:53.620463 13302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:53.623798 13302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "6b3098690a7147939ac1e74c78d89f26"
format_stamp: "Formatted at 2026-09-26 15:02:53 on dist-test-slave-7153"
server_key: "07c6efbc36773f7385c718f3c4c7d167"
server_key_iv: "dd8af636d7eb79d06a072af39077d081"
server_key_version: "encryptionkey@0"
I20260926 15:02:53.624635 13302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "6b3098690a7147939ac1e74c78d89f26"
format_stamp: "Formatted at 2026-09-26 15:02:53 on dist-test-slave-7153"
server_key: "07c6efbc36773f7385c718f3c4c7d167"
server_key_iv: "dd8af636d7eb79d06a072af39077d081"
server_key_version: "encryptionkey@0"
I20260926 15:02:53.630393 13302 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 15:02:53.634117 13317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:53.635601 13302 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:02:53.635792 13302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "6b3098690a7147939ac1e74c78d89f26"
format_stamp: "Formatted at 2026-09-26 15:02:53 on dist-test-slave-7153"
server_key: "07c6efbc36773f7385c718f3c4c7d167"
server_key_iv: "dd8af636d7eb79d06a072af39077d081"
server_key_version: "encryptionkey@0"
I20260926 15:02:53.635986 13302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:53.653196 13302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:53.657691 13302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:53.657959 13302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:53.675683 13302 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:44329
I20260926 15:02:53.675712 13368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:44329 every 8 connection(s)
I20260926 15:02:53.677088 13302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:02:53.681051 13369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:53.687541 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 13302
I20260926 15:02:53.687789 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:02:53.688591 9709 external_mini_cluster.cc:1428] Setting key 2decc5961c5d1559afed32d9eeedfb4d
I20260926 15:02:53.693277 13369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b3098690a7147939ac1e74c78d89f26: Bootstrap starting.
I20260926 15:02:53.697259 13369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b3098690a7147939ac1e74c78d89f26: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:53.698410 13369 log.cc:824] T 00000000000000000000000000000000 P 6b3098690a7147939ac1e74c78d89f26: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:53.701675 13369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b3098690a7147939ac1e74c78d89f26: No bootstrap required, opened a new log
Sep 26 15:02:53 dist-test-slave-7153 krb5kdc[13289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:53.711179 13369 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b3098690a7147939ac1e74c78d89f26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b3098690a7147939ac1e74c78d89f26" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 44329 } }
I20260926 15:02:53.711678 13369 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b3098690a7147939ac1e74c78d89f26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:53.711865 13369 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b3098690a7147939ac1e74c78d89f26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b3098690a7147939ac1e74c78d89f26, State: Initialized, Role: FOLLOWER
I20260926 15:02:53.712570 13369 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b3098690a7147939ac1e74c78d89f26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b3098690a7147939ac1e74c78d89f26" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 44329 } }
I20260926 15:02:53.712832 13369 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b3098690a7147939ac1e74c78d89f26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:02:53.712965 13369 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b3098690a7147939ac1e74c78d89f26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:02:53.713186 13369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b3098690a7147939ac1e74c78d89f26 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:53.713832 13372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:53.690978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46830 (local address 127.9.123.126:44329)
0926 15:02:53.691874 (+ 896us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:53.691888 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:53.691935 (+ 47us) server_negotiation.cc:407] Connection header received
0926 15:02:53.693797 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:53.693829 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:53.694283 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:53.694666 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:53.696568 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:53.698101 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:53.699690 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:53.700380 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:53.700511 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:53.705455 (+ 4944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:53.705464 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:53.705482 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:53.705537 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:53.708833 (+ 3296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:53.709839 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:53.709848 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:53.709856 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:53.710069 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:53.710515 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:53.710519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:53.710521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:53.711206 (+ 685us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:53.711447 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:53.711994 (+ 547us) server_negotiation.cc:299] Negotiation successful
0926 15:02:53.712508 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":426,"threads_started":1}
I20260926 15:02:53.715605 13369 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b3098690a7147939ac1e74c78d89f26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b3098690a7147939ac1e74c78d89f26" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 44329 } }
I20260926 15:02:53.716173 13369 leader_election.cc:302] T 00000000000000000000000000000000 P 6b3098690a7147939ac1e74c78d89f26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b3098690a7147939ac1e74c78d89f26; no voters:
I20260926 15:02:53.716792 13369 leader_election.cc:288] T 00000000000000000000000000000000 P 6b3098690a7147939ac1e74c78d89f26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:02:53.716917 13374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b3098690a7147939ac1e74c78d89f26 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:53.717604 13374 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b3098690a7147939ac1e74c78d89f26 [term 1 LEADER]: Becoming Leader. State: Replica: 6b3098690a7147939ac1e74c78d89f26, State: Running, Role: LEADER
I20260926 15:02:53.718175 13374 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b3098690a7147939ac1e74c78d89f26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b3098690a7147939ac1e74c78d89f26" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 44329 } }
I20260926 15:02:53.719189 13369 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b3098690a7147939ac1e74c78d89f26 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:02:53.721289 13375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b3098690a7147939ac1e74c78d89f26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b3098690a7147939ac1e74c78d89f26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b3098690a7147939ac1e74c78d89f26" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 44329 } } }
I20260926 15:02:53.721556 13376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b3098690a7147939ac1e74c78d89f26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b3098690a7147939ac1e74c78d89f26. Latest consensus state: current_term: 1 leader_uuid: "6b3098690a7147939ac1e74c78d89f26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b3098690a7147939ac1e74c78d89f26" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 44329 } } }
I20260926 15:02:53.721906 13375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b3098690a7147939ac1e74c78d89f26 [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:53.722000 13376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b3098690a7147939ac1e74c78d89f26 [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:53.726672 13382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:02:53.732436 13382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:02:53.743067 13382 catalog_manager.cc:1347] Generated new cluster ID: d1d2ef92844645f8b3360792be09e12a
I20260926 15:02:53.743194 13382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:02:53.771005 13382 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:02:53.772414 13382 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:02:53.785328 13382 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b3098690a7147939ac1e74c78d89f26: Generated new TSK 0
I20260926 15:02:53.786253 13382 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.123.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
I20260926 15:02:53.836676 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:44329
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:54.171543 13395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:54.171871 13395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:54.172003 13395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:54.179399 13395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:54.179543 13395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:54.179708 13395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:02:54.189631 13395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:44329
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:54.190948 13395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:54.192265 13395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:54.205624 13400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:54.205685 13401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:54.205716 13403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:54.206391 13395 server_base.cc:1034] running on GCE node
I20260926 15:02:54.207085 13395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:54.208117 13395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:54.209321 13395 hybrid_clock.cc:648] HybridClock initialized: now 1790434974209263 us; error 51 us; skew 500 ppm
I20260926 15:02:54.209651 13395 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:54 dist-test-slave-7153 krb5kdc[13289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434974, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:54.212735 13395 init.cc:377] Logged in from keytab as kudu/127.9.123.65@KRBTEST.COM (short username kudu)
I20260926 15:02:54.214416 13395 webserver.cc:460] Webserver started at http://127.9.123.65:35579/ using document root <none> and password file <none>
I20260926 15:02:54.215145 13395 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:54.215253 13395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:54.215523 13395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:54.218595 13395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "ea9571a6b1074b1bb14da62ceb35413a"
format_stamp: "Formatted at 2026-09-26 15:02:54 on dist-test-slave-7153"
server_key: "50b2f74a64c6bf59bf0b9498a40b7371"
server_key_iv: "cd82c36935be51bc62e01dbeafabbf65"
server_key_version: "encryptionkey@0"
I20260926 15:02:54.219377 13395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "ea9571a6b1074b1bb14da62ceb35413a"
format_stamp: "Formatted at 2026-09-26 15:02:54 on dist-test-slave-7153"
server_key: "50b2f74a64c6bf59bf0b9498a40b7371"
server_key_iv: "cd82c36935be51bc62e01dbeafabbf65"
server_key_version: "encryptionkey@0"
I20260926 15:02:54.224757 13395 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 15:02:54.228685 13410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:54.230260 13395 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:02:54.230424 13395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "ea9571a6b1074b1bb14da62ceb35413a"
format_stamp: "Formatted at 2026-09-26 15:02:54 on dist-test-slave-7153"
server_key: "50b2f74a64c6bf59bf0b9498a40b7371"
server_key_iv: "cd82c36935be51bc62e01dbeafabbf65"
server_key_version: "encryptionkey@0"
I20260926 15:02:54.230610 13395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:54.251729 13395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:54.256130 13395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:54.256467 13395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:54.257596 13395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:02:54.259025 13395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:54.259105 13395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:54.259192 13395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:54.259249 13395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:54.288861 13395 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:40111
I20260926 15:02:54.288932 13522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:40111 every 8 connection(s)
I20260926 15:02:54.290192 13395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:02:54.297523 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 13395
I20260926 15:02:54.297745 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:02:54.298424 9709 external_mini_cluster.cc:1428] Setting key 7a98dd604eec95739521beb28e21595b
Sep 26 15:02:54 dist-test-slave-7153 krb5kdc[13289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434974, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:54.312017 13525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:54.292612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:36635 (local address 127.9.123.126:44329)
0926 15:02:54.293011 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:54.293015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:54.294136 (+ 1121us) server_negotiation.cc:407] Connection header received
0926 15:02:54.295970 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:54.295977 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:54.296070 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:54.296164 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:54.298082 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:54.298838 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:54.300665 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:54.301045 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:54.301199 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:54.305510 (+ 4311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:54.305517 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:54.305520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:54.305558 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:54.308774 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:54.309677 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:54.309691 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:54.309694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:54.309841 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:54.310247 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:54.310251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:54.310255 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:54.310585 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:54.310733 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:54.311463 (+ 730us) server_negotiation.cc:299] Negotiation successful
0926 15:02:54.311680 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":209,"threads_started":1}
I20260926 15:02:54.312907 13526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:54.293206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:44329 (local address 127.9.123.65:36635)
0926 15:02:54.293966 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:54.294006 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:54.295672 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:54.296428 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:54.296437 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:54.296988 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:54.297884 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:54.297913 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:54.298986 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:54.298990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:54.300534 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:54.300548 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:54.301297 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:54.301304 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:54.301605 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:54.302314 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:54.302348 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:54.305231 (+ 2883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:54.308958 (+ 3727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:54.308966 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:54.308984 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:54.309531 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:54.309945 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:54.309949 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:54.309951 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:54.310147 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:54.310710 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:54.310716 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:54.311095 (+ 379us) client_negotiation.cc:769] Sending connection context
0926 15:02:54.311329 (+ 234us) client_negotiation.cc:240] Negotiation successful
0926 15:02:54.311753 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":281,"threads_started":1}
I20260926 15:02:54.314509 13523 heartbeater.cc:347] Connected to a master server at 127.9.123.126:44329
I20260926 15:02:54.314931 13523 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:54.315893 13523 heartbeater.cc:511] Master 127.9.123.126:44329 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.123.66@KRBTEST.COM; defaulting to no policy
I20260926 15:02:54.318377 13334 ts_manager.cc:194] Registered new tserver with Master: ea9571a6b1074b1bb14da62ceb35413a (127.9.123.65:40111)
I20260926 15:02:54.319972 13334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.65@KRBTEST.COM'} at 127.9.123.65:36635
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
I20260926 15:02:54.344705 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:44329
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:54.666178 13529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:54.666488 13529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:54.666612 13529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:54.673934 13529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:54.674065 13529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:54.674213 13529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:02:54.684343 13529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:44329
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:54.685750 13529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:54.687081 13529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:54.699607 13535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:54.699657 13537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:54.699635 13534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:54.700341 13529 server_base.cc:1034] running on GCE node
I20260926 15:02:54.700937 13529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:54.701901 13529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:54.703114 13529 hybrid_clock.cc:648] HybridClock initialized: now 1790434974703060 us; error 44 us; skew 500 ppm
I20260926 15:02:54.703420 13529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:54 dist-test-slave-7153 krb5kdc[13289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434974, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:54.706529 13529 init.cc:377] Logged in from keytab as kudu/127.9.123.66@KRBTEST.COM (short username kudu)
I20260926 15:02:54.708166 13529 webserver.cc:460] Webserver started at http://127.9.123.66:38433/ using document root <none> and password file <none>
I20260926 15:02:54.709002 13529 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:54.709156 13529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:54.709492 13529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:54.712468 13529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "16d98cc6933446ee931cfeff8bad3e85"
format_stamp: "Formatted at 2026-09-26 15:02:54 on dist-test-slave-7153"
server_key: "55933b13204e0d8e1da653104dd4ad2b"
server_key_iv: "bf7a70e3ab170cf93d64ddc130a3410c"
server_key_version: "encryptionkey@0"
I20260926 15:02:54.713279 13529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "16d98cc6933446ee931cfeff8bad3e85"
format_stamp: "Formatted at 2026-09-26 15:02:54 on dist-test-slave-7153"
server_key: "55933b13204e0d8e1da653104dd4ad2b"
server_key_iv: "bf7a70e3ab170cf93d64ddc130a3410c"
server_key_version: "encryptionkey@0"
I20260926 15:02:54.718446 13529 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 15:02:54.722113 13544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:54.723517 13529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 15:02:54.723678 13529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "16d98cc6933446ee931cfeff8bad3e85"
format_stamp: "Formatted at 2026-09-26 15:02:54 on dist-test-slave-7153"
server_key: "55933b13204e0d8e1da653104dd4ad2b"
server_key_iv: "bf7a70e3ab170cf93d64ddc130a3410c"
server_key_version: "encryptionkey@0"
I20260926 15:02:54.723883 13529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:54.751520 13529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:54.755613 13529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:54.755887 13529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:54.756827 13529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:02:54.758291 13529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:54.758363 13529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:54.758468 13529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:54.758514 13529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:54.788161 13529 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:34525
I20260926 15:02:54.788195 13656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:34525 every 8 connection(s)
I20260926 15:02:54.789439 13529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:02:54.793118 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 13529
I20260926 15:02:54.793314 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:02:54.793780 9709 external_mini_cluster.cc:1428] Setting key 7fb911390a6427a4378c793a67fe8701
Sep 26 15:02:54 dist-test-slave-7153 krb5kdc[13289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434974, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:54.810437 13525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:54.791834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:35593 (local address 127.9.123.126:44329)
0926 15:02:54.792036 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:54.792042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:54.793256 (+ 1214us) server_negotiation.cc:407] Connection header received
0926 15:02:54.795407 (+ 2151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:54.795412 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:54.795487 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:54.795600 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:54.797565 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:54.798257 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:54.800348 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:54.800951 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:54.801055 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:54.804617 (+ 3562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:54.804624 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:54.804628 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:54.804670 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:54.807298 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:54.808140 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:54.808144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:54.808146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:54.808275 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:54.808705 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:54.808708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:54.808710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:54.808990 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:54.809141 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:54.809876 (+ 735us) server_negotiation.cc:299] Negotiation successful
0926 15:02:54.810104 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
WARNING: no policy specified for kudu/127.9.123.67@KRBTEST.COM; defaulting to no policy
I20260926 15:02:54.811463 13659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:54.792381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:44329 (local address 127.9.123.66:35593)
0926 15:02:54.793070 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:54.793149 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:54.795129 (+ 1980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:54.795836 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:54.795847 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:54.796577 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:54.797402 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:54.797428 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:54.798397 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:54.798403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:54.800163 (+ 1760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:54.800192 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:54.801210 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:54.801218 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:54.801443 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:54.802137 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:54.802161 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:54.804353 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:54.807480 (+ 3127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:54.807491 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:54.807512 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:54.807999 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:54.808383 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:54.808388 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:54.808391 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:54.808587 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:54.809142 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:54.809151 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:54.809514 (+ 363us) client_negotiation.cc:769] Sending connection context
0926 15:02:54.809758 (+ 244us) client_negotiation.cc:240] Negotiation successful
0926 15:02:54.810306 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":260,"threads_started":1}
I20260926 15:02:54.813138 13657 heartbeater.cc:347] Connected to a master server at 127.9.123.126:44329
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.67@KRBTEST.COM" created.
I20260926 15:02:54.813480 13657 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:54.814242 13657 heartbeater.cc:511] Master 127.9.123.126:44329 requested a full tablet report, sending...
I20260926 15:02:54.815748 13334 ts_manager.cc:194] Registered new tserver with Master: 16d98cc6933446ee931cfeff8bad3e85 (127.9.123.66:34525)
I20260926 15:02:54.816572 13334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.66@KRBTEST.COM'} at 127.9.123.66:35593
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
I20260926 15:02:54.837510 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:44329
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:33723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:55.151795 13662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:55.152107 13662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:55.152240 13662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:55.159575 13662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:55.159713 13662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:55.159873 13662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:02:55.169523 13662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:33723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:44329
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:55.170758 13662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:55.172037 13662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:55.184021 13667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:55.184108 13668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:55.184053 13670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:55.184940 13662 server_base.cc:1034] running on GCE node
I20260926 15:02:55.185514 13662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:55.186376 13662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:55.187568 13662 hybrid_clock.cc:648] HybridClock initialized: now 1790434975187520 us; error 35 us; skew 500 ppm
I20260926 15:02:55.187860 13662 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:55 dist-test-slave-7153 krb5kdc[13289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434975, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:55.190814 13662 init.cc:377] Logged in from keytab as kudu/127.9.123.67@KRBTEST.COM (short username kudu)
I20260926 15:02:55.192281 13662 webserver.cc:460] Webserver started at http://127.9.123.67:38885/ using document root <none> and password file <none>
I20260926 15:02:55.192952 13662 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:55.193069 13662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:55.193385 13662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:55.196219 13662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "be0a5092faee4d9bbf1f414e32f53c5c"
format_stamp: "Formatted at 2026-09-26 15:02:55 on dist-test-slave-7153"
server_key: "b1d2ad6823b3c3ff33d71c37ea5fcea4"
server_key_iv: "42989f254dd5140da0321b2000e1e6e8"
server_key_version: "encryptionkey@0"
I20260926 15:02:55.196956 13662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "be0a5092faee4d9bbf1f414e32f53c5c"
format_stamp: "Formatted at 2026-09-26 15:02:55 on dist-test-slave-7153"
server_key: "b1d2ad6823b3c3ff33d71c37ea5fcea4"
server_key_iv: "42989f254dd5140da0321b2000e1e6e8"
server_key_version: "encryptionkey@0"
I20260926 15:02:55.202090 13662 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260926 15:02:55.206000 13677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:55.207306 13662 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:02:55.207446 13662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "be0a5092faee4d9bbf1f414e32f53c5c"
format_stamp: "Formatted at 2026-09-26 15:02:55 on dist-test-slave-7153"
server_key: "b1d2ad6823b3c3ff33d71c37ea5fcea4"
server_key_iv: "42989f254dd5140da0321b2000e1e6e8"
server_key_version: "encryptionkey@0"
I20260926 15:02:55.207626 13662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:55.234345 13662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:55.238847 13662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:55.239114 13662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:55.240074 13662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:02:55.241389 13662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:55.241462 13662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:55.241576 13662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:55.241626 13662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:55.271875 13662 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:35471
I20260926 15:02:55.271910 13789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:35471 every 8 connection(s)
I20260926 15:02:55.273140 13662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:02:55.274927 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 13662
I20260926 15:02:55.275151 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:02:55.275566 9709 external_mini_cluster.cc:1428] Setting key 9bf887420999e9d519fd361dc075e48e
Sep 26 15:02:55 dist-test-slave-7153 krb5kdc[13289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434975, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:55.292490 13525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:55.275800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:51249 (local address 127.9.123.126:44329)
0926 15:02:55.276009 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:55.276015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:55.277242 (+ 1227us) server_negotiation.cc:407] Connection header received
0926 15:02:55.279046 (+ 1804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:55.279051 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:55.279130 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:55.279208 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:55.280695 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:55.281464 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:55.282955 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:55.283329 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:55.283435 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:55.286775 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:55.286780 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:55.286783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:55.286823 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:55.289332 (+ 2509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:55.290222 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:55.290227 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:55.290231 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:55.290383 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:55.290847 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:55.290850 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:55.290853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:55.291154 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:55.291266 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:55.292047 (+ 781us) server_negotiation.cc:299] Negotiation successful
0926 15:02:55.292242 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260926 15:02:55.293912 13792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:55.276379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:44329 (local address 127.9.123.67:51249)
0926 15:02:55.277076 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:55.277152 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:55.278794 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:55.279450 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:55.279458 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:55.279965 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:55.280541 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:55.280558 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:55.281610 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:55.281614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:55.282819 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:55.282834 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:55.283560 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:55.283569 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:55.283783 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:55.284433 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:55.284454 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:55.286555 (+ 2101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:55.289488 (+ 2933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:55.289498 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:55.289517 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:55.290112 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:55.290488 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:55.290492 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:55.290495 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:55.290723 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:55.291280 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:55.291288 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:55.291680 (+ 392us) client_negotiation.cc:769] Sending connection context
0926 15:02:55.291987 (+ 307us) client_negotiation.cc:240] Negotiation successful
0926 15:02:55.292554 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":287,"threads_started":1}
I20260926 15:02:55.295220 13790 heartbeater.cc:347] Connected to a master server at 127.9.123.126:44329
I20260926 15:02:55.295578 13790 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:55.296293 13790 heartbeater.cc:511] Master 127.9.123.126:44329 requested a full tablet report, sending...
I20260926 15:02:55.297683 13334 ts_manager.cc:194] Registered new tserver with Master: be0a5092faee4d9bbf1f414e32f53c5c (127.9.123.67:35471)
I20260926 15:02:55.298465 13334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.67@KRBTEST.COM'} at 127.9.123.67:51249
I20260926 15:02:55.302830 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 15:02:55.323439 13523 heartbeater.cc:503] Master 127.9.123.126:44329 was elected leader, sending a full tablet report...
Sep 26 15:02:55 dist-test-slave-7153 krb5kdc[13289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 15:02:55 dist-test-slave-7153 krb5kdc[13289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.65@KRBTEST.COM
I20260926 15:02:55.347752 13797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:55.333650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43336 (local address 127.9.123.65:40111)
0926 15:02:55.334220 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:55.334227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:55.334253 (+ 26us) server_negotiation.cc:407] Connection header received
0926 15:02:55.334334 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:55.334341 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:55.334569 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:55.334702 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:55.336193 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:55.337242 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:55.338453 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:55.338996 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:55.339085 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:55.342797 (+ 3712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:55.342808 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:55.342823 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:55.342859 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:55.345509 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:55.346064 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:55.346071 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:55.346073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:55.346182 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:55.346444 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:55.346446 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:55.346448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:55.346770 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:55.346983 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:55.347189 (+ 206us) server_negotiation.cc:299] Negotiation successful
0926 15:02:55.347391 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":295,"threads_started":1}
Sep 26 15:02:55 dist-test-slave-7153 krb5kdc[13289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:55.364845 13525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:55.351652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46840 (local address 127.9.123.126:44329)
0926 15:02:55.351909 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:55.351915 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:55.352006 (+ 91us) server_negotiation.cc:407] Connection header received
0926 15:02:55.352130 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:55.352134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:55.352194 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:55.352278 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:55.353590 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:55.354339 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:55.355585 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:55.356011 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:55.356112 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:55.359924 (+ 3812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:55.359929 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:55.359932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:55.359972 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:55.362480 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:55.363149 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:55.363153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:55.363156 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:55.363305 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:55.363643 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:55.363646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:55.363648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:55.363968 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:55.364135 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:55.364436 (+ 301us) server_negotiation.cc:299] Negotiation successful
0926 15:02:55.364607 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
W20260926 15:02:55.365439 13334 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46840
I20260926 15:02:55.366398 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 13395
I20260926 15:02:55.378980 13797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:55.367330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43350 (local address 127.9.123.65:40111)
0926 15:02:55.367515 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:55.367521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:55.367694 (+ 173us) server_negotiation.cc:407] Connection header received
0926 15:02:55.367875 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:55.367879 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:55.367969 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:55.368049 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:55.369245 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:55.369988 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:55.371303 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:55.371723 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:55.371838 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:55.374141 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:55.374146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:55.374149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:55.374184 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:55.376596 (+ 2412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:55.377217 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:55.377223 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:55.377226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:55.377357 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:55.377710 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:55.377713 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:55.377715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:55.377969 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:55.378178 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:55.378485 (+ 307us) server_negotiation.cc:299] Negotiation successful
0926 15:02:55.378692 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 15:02:55.379563 13518 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:02:55.460261 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 13395
I20260926 15:02:55.473028 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 13529
Sep 26 15:02:55 dist-test-slave-7153 krb5kdc[13289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.66@KRBTEST.COM
I20260926 15:02:55.488464 13802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:55.473982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58700 (local address 127.9.123.66:34525)
0926 15:02:55.474543 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:55.474550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:55.474576 (+ 26us) server_negotiation.cc:407] Connection header received
0926 15:02:55.474634 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:55.474641 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:55.474857 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:55.474982 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:55.476123 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:55.477143 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:55.478364 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:55.478904 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:55.479003 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:55.482875 (+ 3872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:55.482888 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:55.482903 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:55.482937 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:55.485623 (+ 2686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:55.486255 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:55.486262 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:55.486264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:55.486395 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:55.486770 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:55.486774 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:55.486776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:55.487275 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:55.487581 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:55.487888 (+ 307us) server_negotiation.cc:299] Negotiation successful
0926 15:02:55.488135 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":294,"threads_started":1}
I20260926 15:02:55.489768 13652 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:02:55.573848 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 13529
I20260926 15:02:55.586448 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 13662
Sep 26 15:02:55 dist-test-slave-7153 krb5kdc[13289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.67@KRBTEST.COM
I20260926 15:02:55.602149 13804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:55.587354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44198 (local address 127.9.123.67:35471)
0926 15:02:55.587910 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:55.587916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:55.587944 (+ 28us) server_negotiation.cc:407] Connection header received
0926 15:02:55.588000 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:55.588006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:55.588205 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:55.588329 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:55.589524 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:55.590493 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:55.591769 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:55.592342 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:55.592464 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:55.596771 (+ 4307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:55.596785 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:55.596800 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:55.596861 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:55.599589 (+ 2728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:55.600242 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:55.600248 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:55.600250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:55.600364 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:55.600639 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:55.600642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:55.600644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:55.600978 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:55.601237 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:55.601577 (+ 340us) server_negotiation.cc:299] Negotiation successful
0926 15:02:55.601785 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":260,"threads_started":1}
I20260926 15:02:55.603438 13785 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:02:55.685277 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 13662
I20260926 15:02:55.697371 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 13302
I20260926 15:02:55.698474 13364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:02:55.798761 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 13302
2026-09-26T15:02:55Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3746 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:02:55 dist-test-slave-7153 krb5kdc[13809](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:02:55 dist-test-slave-7153 krb5kdc[13809](info): set up 2 sockets
Sep 26 15:02:55 dist-test-slave-7153 krb5kdc[13809](info): commencing operation
krb5kdc: starting...
W20260926 15:02:56.801463 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.961s user 0.001s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:02:56 dist-test-slave-7153 krb5kdc[13809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:02:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:02:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:02:57.000897 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:42813
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:36625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:42813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:57.321760 13822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:57.322089 13822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:57.322209 13822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:57.329505 13822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:02:57.329619 13822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:57.329677 13822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:57.329751 13822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:02:57.329793 13822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:02:57.339681 13822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:36625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:42813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:42813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:57.340998 13822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:57.342296 13822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:57.353941 13828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:57.353999 13827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:57.353960 13830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:57.354791 13822 server_base.cc:1034] running on GCE node
I20260926 15:02:57.355546 13822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:57.356966 13822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:57.358181 13822 hybrid_clock.cc:648] HybridClock initialized: now 1790434977358133 us; error 61 us; skew 500 ppm
I20260926 15:02:57.358532 13822 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:57 dist-test-slave-7153 krb5kdc[13809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434977, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:57.366170 13822 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:02:57.367412 13822 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:57.367523 13822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:57.367835 13822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:57.370916 13822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "fd71bca671bb49a4a5fbd031480cfd67"
format_stamp: "Formatted at 2026-09-26 15:02:57 on dist-test-slave-7153"
server_key: "767615bd73d5a6e746fa44d595dc6432"
server_key_iv: "35d9235ac9b75d354915bc8571ac9dfa"
server_key_version: "encryptionkey@0"
I20260926 15:02:57.371698 13822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "fd71bca671bb49a4a5fbd031480cfd67"
format_stamp: "Formatted at 2026-09-26 15:02:57 on dist-test-slave-7153"
server_key: "767615bd73d5a6e746fa44d595dc6432"
server_key_iv: "35d9235ac9b75d354915bc8571ac9dfa"
server_key_version: "encryptionkey@0"
I20260926 15:02:57.376971 13822 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 15:02:57.380519 13836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:57.381968 13822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:02:57.382145 13822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "fd71bca671bb49a4a5fbd031480cfd67"
format_stamp: "Formatted at 2026-09-26 15:02:57 on dist-test-slave-7153"
server_key: "767615bd73d5a6e746fa44d595dc6432"
server_key_iv: "35d9235ac9b75d354915bc8571ac9dfa"
server_key_version: "encryptionkey@0"
I20260926 15:02:57.382349 13822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:57.414685 13822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:57.419015 13822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:57.419278 13822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:57.436192 13822 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:42813
I20260926 15:02:57.436172 13887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:42813 every 8 connection(s)
I20260926 15:02:57.437140 13822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:02:57.439204 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 13822
I20260926 15:02:57.439443 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:02:57.440245 9709 external_mini_cluster.cc:1428] Setting key 5c5c3f9759ff8ccd6cd06effbff64e18
I20260926 15:02:57.441529 13888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:57.453419 13888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd71bca671bb49a4a5fbd031480cfd67: Bootstrap starting.
Sep 26 15:02:57 dist-test-slave-7153 krb5kdc[13809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:57.456750 13888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd71bca671bb49a4a5fbd031480cfd67: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:57.457834 13888 log.cc:824] T 00000000000000000000000000000000 P fd71bca671bb49a4a5fbd031480cfd67: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:57.460521 13888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd71bca671bb49a4a5fbd031480cfd67: No bootstrap required, opened a new log
I20260926 15:02:57.461738 13891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:57.442419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48458 (local address 127.9.123.126:42813)
0926 15:02:57.443230 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:57.443246 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:57.443288 (+ 42us) server_negotiation.cc:407] Connection header received
0926 15:02:57.444812 (+ 1524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:57.444836 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:57.445344 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:57.445769 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:57.447224 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:57.448514 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:57.449803 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:57.450487 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:57.450660 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:57.454553 (+ 3893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:57.454561 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:57.454576 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:57.454614 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:57.457734 (+ 3120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:57.458448 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:57.458454 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:57.458460 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:57.458634 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:57.458989 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:57.458993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:57.458995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:57.459525 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:57.459804 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:57.460219 (+ 415us) server_negotiation.cc:299] Negotiation successful
0926 15:02:57.460650 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":384,"threads_started":1}
I20260926 15:02:57.468505 13888 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd71bca671bb49a4a5fbd031480cfd67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd71bca671bb49a4a5fbd031480cfd67" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42813 } }
I20260926 15:02:57.468991 13888 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd71bca671bb49a4a5fbd031480cfd67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:57.469125 13888 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd71bca671bb49a4a5fbd031480cfd67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd71bca671bb49a4a5fbd031480cfd67, State: Initialized, Role: FOLLOWER
I20260926 15:02:57.469743 13888 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd71bca671bb49a4a5fbd031480cfd67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd71bca671bb49a4a5fbd031480cfd67" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42813 } }
I20260926 15:02:57.469960 13888 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd71bca671bb49a4a5fbd031480cfd67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:02:57.470088 13888 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd71bca671bb49a4a5fbd031480cfd67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:02:57.470237 13888 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd71bca671bb49a4a5fbd031480cfd67 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:57.472380 13888 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd71bca671bb49a4a5fbd031480cfd67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd71bca671bb49a4a5fbd031480cfd67" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42813 } }
I20260926 15:02:57.472911 13888 leader_election.cc:302] T 00000000000000000000000000000000 P fd71bca671bb49a4a5fbd031480cfd67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd71bca671bb49a4a5fbd031480cfd67; no voters:
I20260926 15:02:57.473546 13888 leader_election.cc:288] T 00000000000000000000000000000000 P fd71bca671bb49a4a5fbd031480cfd67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:02:57.473677 13893 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd71bca671bb49a4a5fbd031480cfd67 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:57.474455 13893 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd71bca671bb49a4a5fbd031480cfd67 [term 1 LEADER]: Becoming Leader. State: Replica: fd71bca671bb49a4a5fbd031480cfd67, State: Running, Role: LEADER
I20260926 15:02:57.474995 13893 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd71bca671bb49a4a5fbd031480cfd67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd71bca671bb49a4a5fbd031480cfd67" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42813 } }
I20260926 15:02:57.475709 13888 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd71bca671bb49a4a5fbd031480cfd67 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:02:57.477844 13894 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd71bca671bb49a4a5fbd031480cfd67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd71bca671bb49a4a5fbd031480cfd67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd71bca671bb49a4a5fbd031480cfd67" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42813 } } }
I20260926 15:02:57.478186 13894 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd71bca671bb49a4a5fbd031480cfd67 [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:57.478742 13895 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd71bca671bb49a4a5fbd031480cfd67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd71bca671bb49a4a5fbd031480cfd67. Latest consensus state: current_term: 1 leader_uuid: "fd71bca671bb49a4a5fbd031480cfd67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd71bca671bb49a4a5fbd031480cfd67" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42813 } } }
I20260926 15:02:57.479125 13895 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd71bca671bb49a4a5fbd031480cfd67 [sys.catalog]: This master's current role is: LEADER
I20260926 15:02:57.480902 13898 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:02:57.485921 13898 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:02:57.495685 13898 catalog_manager.cc:1347] Generated new cluster ID: 911da4efedd040c9be312983861b3a9d
I20260926 15:02:57.495800 13898 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:02:57.514480 13898 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:02:57.516505 13898 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:02:57.528625 13898 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd71bca671bb49a4a5fbd031480cfd67: Generated new TSK 0
I20260926 15:02:57.529800 13898 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.123.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
I20260926 15:02:57.596092 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:42813
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:36625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:57.935575 13914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:57.935878 13914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:57.935976 13914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:57.943331 13914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:57.943455 13914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:57.943570 13914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:02:57.953518 13914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:36625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:42813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:57.954829 13914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:57.956137 13914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:57.969676 13922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:57.969652 13920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:57.969973 13919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:57.971676 13914 server_base.cc:1034] running on GCE node
I20260926 15:02:57.972399 13914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:57.973457 13914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:57.974718 13914 hybrid_clock.cc:648] HybridClock initialized: now 1790434977974662 us; error 51 us; skew 500 ppm
I20260926 15:02:57.975097 13914 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:57 dist-test-slave-7153 krb5kdc[13809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434977, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:57.978588 13914 init.cc:377] Logged in from keytab as kudu/127.9.123.65@KRBTEST.COM (short username kudu)
I20260926 15:02:57.979874 13914 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:57.979988 13914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:57.980401 13914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:57.983538 13914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "ef992cab50374dc9a4c59e59109d2d95"
format_stamp: "Formatted at 2026-09-26 15:02:57 on dist-test-slave-7153"
server_key: "ad702f0f34f37e2e2680382b516e1f04"
server_key_iv: "52afa6f24886686255252c260b05f629"
server_key_version: "encryptionkey@0"
I20260926 15:02:57.984326 13914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "ef992cab50374dc9a4c59e59109d2d95"
format_stamp: "Formatted at 2026-09-26 15:02:57 on dist-test-slave-7153"
server_key: "ad702f0f34f37e2e2680382b516e1f04"
server_key_iv: "52afa6f24886686255252c260b05f629"
server_key_version: "encryptionkey@0"
I20260926 15:02:57.989724 13914 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 15:02:57.993556 13928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:57.995036 13914 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 15:02:57.995204 13914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "ef992cab50374dc9a4c59e59109d2d95"
format_stamp: "Formatted at 2026-09-26 15:02:57 on dist-test-slave-7153"
server_key: "ad702f0f34f37e2e2680382b516e1f04"
server_key_iv: "52afa6f24886686255252c260b05f629"
server_key_version: "encryptionkey@0"
I20260926 15:02:57.995426 13914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:58.017303 13914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:58.021924 13914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:58.022193 13914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:58.023244 13914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:02:58.024657 13914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:58.024747 13914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:58.024860 13914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:58.024927 13914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:58.055351 13914 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:34153
I20260926 15:02:58.055382 14040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:34153 every 8 connection(s)
I20260926 15:02:58.056206 13914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:02:58.058329 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 13914
I20260926 15:02:58.058548 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:02:58.059239 9709 external_mini_cluster.cc:1428] Setting key 875a05251ed954040caa12017b44352e
Sep 26 15:02:58 dist-test-slave-7153 krb5kdc[13809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434977, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.123.66@KRBTEST.COM; defaulting to no policy
I20260926 15:02:58.078297 14043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:58.058693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:60163 (local address 127.9.123.126:42813)
0926 15:02:58.059207 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:58.059213 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:58.060253 (+ 1040us) server_negotiation.cc:407] Connection header received
0926 15:02:58.062246 (+ 1993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:58.062252 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:58.062340 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:58.062430 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:58.064002 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:58.064733 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:58.066847 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:58.067240 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:58.067353 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:58.071129 (+ 3776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:58.071136 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:58.071141 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:58.071186 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:58.074582 (+ 3396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:58.075557 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:58.075563 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:58.075567 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:58.075737 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:58.076373 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:58.076378 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:58.076381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:58.076759 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:58.076912 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:58.077803 (+ 891us) server_negotiation.cc:299] Negotiation successful
0926 15:02:58.078007 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":248,"threads_started":1}
I20260926 15:02:58.079435 14044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:58.059270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:42813 (local address 127.9.123.65:60163)
0926 15:02:58.060095 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:58.060149 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:58.061958 (+ 1809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:58.062695 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:58.062705 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:58.063223 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:58.063826 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:58.063845 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:58.064904 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:58.064908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:58.066680 (+ 1772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:58.066695 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:58.067462 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:58.067468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:58.067686 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:58.068384 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:58.068407 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:58.070851 (+ 2444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:58.074811 (+ 3960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:58.074822 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:58.074839 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:58.075379 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:58.075918 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:58.075923 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:58.075926 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:58.076229 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:58.076934 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:58.076946 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:58.077410 (+ 464us) client_negotiation.cc:769] Sending connection context
0926 15:02:58.077677 (+ 267us) client_negotiation.cc:240] Negotiation successful
0926 15:02:58.078204 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":326,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.66@KRBTEST.COM" created.
I20260926 15:02:58.081228 14041 heartbeater.cc:347] Connected to a master server at 127.9.123.126:42813
I20260926 15:02:58.081554 14041 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:58.082262 14041 heartbeater.cc:511] Master 127.9.123.126:42813 requested a full tablet report, sending...
I20260926 15:02:58.084689 13853 ts_manager.cc:194] Registered new tserver with Master: ef992cab50374dc9a4c59e59109d2d95 (127.9.123.65:34153)
I20260926 15:02:58.086393 13853 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.65@KRBTEST.COM'} at 127.9.123.65:60163
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
I20260926 15:02:58.104355 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:42813
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:36625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:58.440567 14047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:58.440874 14047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:58.440999 14047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:58.448546 14047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:58.448681 14047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:58.448834 14047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:02:58.458798 14047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:36625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:42813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:58.460111 14047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:58.461462 14047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:58.474653 14052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:58.474937 14047 server_base.cc:1034] running on GCE node
W20260926 15:02:58.474691 14055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:58.474717 14053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:58.476074 14047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:58.477042 14047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:58.478279 14047 hybrid_clock.cc:648] HybridClock initialized: now 1790434978478212 us; error 49 us; skew 500 ppm
I20260926 15:02:58.478601 14047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:58 dist-test-slave-7153 krb5kdc[13809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434978, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:58.481798 14047 init.cc:377] Logged in from keytab as kudu/127.9.123.66@KRBTEST.COM (short username kudu)
I20260926 15:02:58.482924 14047 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:58.483018 14047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:58.483291 14047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:58.486495 14047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "ba3ff5fbba214909b1a8040e252021d0"
format_stamp: "Formatted at 2026-09-26 15:02:58 on dist-test-slave-7153"
server_key: "49eb4824c4a2acf540f268dd69232fa6"
server_key_iv: "1b7a5d8ec621d650c2675f9e171c8e41"
server_key_version: "encryptionkey@0"
I20260926 15:02:58.487334 14047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "ba3ff5fbba214909b1a8040e252021d0"
format_stamp: "Formatted at 2026-09-26 15:02:58 on dist-test-slave-7153"
server_key: "49eb4824c4a2acf540f268dd69232fa6"
server_key_iv: "1b7a5d8ec621d650c2675f9e171c8e41"
server_key_version: "encryptionkey@0"
I20260926 15:02:58.492722 14047 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 15:02:58.496479 14061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:58.498202 14047 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:02:58.498379 14047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "ba3ff5fbba214909b1a8040e252021d0"
format_stamp: "Formatted at 2026-09-26 15:02:58 on dist-test-slave-7153"
server_key: "49eb4824c4a2acf540f268dd69232fa6"
server_key_iv: "1b7a5d8ec621d650c2675f9e171c8e41"
server_key_version: "encryptionkey@0"
I20260926 15:02:58.498600 14047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:58.513450 14047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:58.517833 14047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:58.518077 14047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:58.519029 14047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:02:58.520450 14047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:58.520539 14047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:58.520627 14047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:58.520685 14047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:58.551008 14047 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:45047
I20260926 15:02:58.551039 14173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:45047 every 8 connection(s)
I20260926 15:02:58.551841 14047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:02:58.553906 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 14047
I20260926 15:02:58.554095 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:02:58.554760 9709 external_mini_cluster.cc:1428] Setting key 63c1620eee8886df6ad842f74309058c
Sep 26 15:02:58 dist-test-slave-7153 krb5kdc[13809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434978, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.123.67@KRBTEST.COM; defaulting to no policy
I20260926 15:02:58.576584 14043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:58.554561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:45631 (local address 127.9.123.126:42813)
0926 15:02:58.554782 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:58.554789 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:58.556332 (+ 1543us) server_negotiation.cc:407] Connection header received
0926 15:02:58.558504 (+ 2172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:58.558511 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:58.558593 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:58.558694 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:58.560670 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:58.561666 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:58.564303 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:58.565056 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:58.565406 (+ 350us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:58.569848 (+ 4442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:58.569854 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:58.569858 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:58.569897 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:58.572453 (+ 2556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:58.573658 (+ 1205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:58.573663 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:58.573666 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:58.573846 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:58.574509 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:58.574514 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:58.574516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:58.574809 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:58.574964 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:58.575943 (+ 979us) server_negotiation.cc:299] Negotiation successful
0926 15:02:58.576173 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.67@KRBTEST.COM" created.
I20260926 15:02:58.577704 14176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:58.555246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:42813 (local address 127.9.123.66:45631)
0926 15:02:58.556158 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:58.556208 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:58.558195 (+ 1987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:58.558927 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:58.559033 (+ 106us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:58.559643 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:58.560310 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:58.560333 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:58.561857 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:58.561862 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:58.564072 (+ 2210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:58.564094 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:58.565594 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:58.565604 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:58.565895 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:58.566686 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:58.566716 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:58.569499 (+ 2783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:58.572664 (+ 3165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:58.572677 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:58.572701 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:58.573483 (+ 782us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:58.574130 (+ 647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:58.574134 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:58.574137 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:58.574345 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:58.575061 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:58.575068 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:58.575517 (+ 449us) client_negotiation.cc:769] Sending connection context
0926 15:02:58.575795 (+ 278us) client_negotiation.cc:240] Negotiation successful
0926 15:02:58.576233 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":329,"threads_started":1}
I20260926 15:02:58.579741 14174 heartbeater.cc:347] Connected to a master server at 127.9.123.126:42813
I20260926 15:02:58.580078 14174 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:58.581030 14174 heartbeater.cc:511] Master 127.9.123.126:42813 requested a full tablet report, sending...
I20260926 15:02:58.582644 13853 ts_manager.cc:194] Registered new tserver with Master: ba3ff5fbba214909b1a8040e252021d0 (127.9.123.66:45047)
I20260926 15:02:58.583735 13853 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.66@KRBTEST.COM'} at 127.9.123.66:45631
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
I20260926 15:02:58.610399 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:42813
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:36625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:02:58.936314 14179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:02:58.936707 14179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:02:58.936846 14179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:02:58.946782 14179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:02:58.946910 14179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:02:58.947067 14179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:02:58.957288 14179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:36625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:42813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:02:58.958695 14179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:02:58.960059 14179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:02:58.973145 14184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:58.973178 14185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:02:58.973248 14187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:02:58.974355 14179 server_base.cc:1034] running on GCE node
I20260926 15:02:58.974978 14179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:02:58.976009 14179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:02:58.977238 14179 hybrid_clock.cc:648] HybridClock initialized: now 1790434978977180 us; error 41 us; skew 500 ppm
I20260926 15:02:58.977592 14179 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:02:58 dist-test-slave-7153 krb5kdc[13809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434978, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:02:58.980683 14179 init.cc:377] Logged in from keytab as kudu/127.9.123.67@KRBTEST.COM (short username kudu)
I20260926 15:02:58.981905 14179 fs_manager.cc:359] Metadata directory not provided
I20260926 15:02:58.982000 14179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:02:58.982303 14179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:02:58.985352 14179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "bd29214bcdd04a4f90b7f18642d74dec"
format_stamp: "Formatted at 2026-09-26 15:02:58 on dist-test-slave-7153"
server_key: "124e2cb226d6048de5f6cd7327ac087a"
server_key_iv: "1867ed6bd1fafdc6b26099d0be65a9ad"
server_key_version: "encryptionkey@0"
I20260926 15:02:58.986279 14179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "bd29214bcdd04a4f90b7f18642d74dec"
format_stamp: "Formatted at 2026-09-26 15:02:58 on dist-test-slave-7153"
server_key: "124e2cb226d6048de5f6cd7327ac087a"
server_key_iv: "1867ed6bd1fafdc6b26099d0be65a9ad"
server_key_version: "encryptionkey@0"
I20260926 15:02:58.991393 14179 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 15:02:58.994935 14193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:58.996284 14179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 15:02:58.996433 14179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "bd29214bcdd04a4f90b7f18642d74dec"
format_stamp: "Formatted at 2026-09-26 15:02:58 on dist-test-slave-7153"
server_key: "124e2cb226d6048de5f6cd7327ac087a"
server_key_iv: "1867ed6bd1fafdc6b26099d0be65a9ad"
server_key_version: "encryptionkey@0"
I20260926 15:02:58.996651 14179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:02:59.014837 14179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:02:59.019286 14179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:02:59.019556 14179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:02:59.020560 14179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:02:59.022054 14179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:02:59.022141 14179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:59.022243 14179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:02:59.022305 14179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:02:59.052222 14179 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:36587
I20260926 15:02:59.052296 14305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:36587 every 8 connection(s)
I20260926 15:02:59.053053 14179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:02:59.059252 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 14179
I20260926 15:02:59.059459 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:02:59.059934 9709 external_mini_cluster.cc:1428] Setting key 386406980cfc2ea7cfdce7590d862250
Sep 26 15:02:59 dist-test-slave-7153 krb5kdc[13809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434978, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:02:59.073706 14043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:59.055708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:47021 (local address 127.9.123.126:42813)
0926 15:02:59.055928 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:59.055935 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:59.057228 (+ 1293us) server_negotiation.cc:407] Connection header received
0926 15:02:59.059758 (+ 2530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:59.059764 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:59.059891 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:59.060020 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:59.061839 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.062686 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.064257 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.064646 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.064740 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:59.068005 (+ 3265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:59.068010 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:59.068013 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:59.068048 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:59.070629 (+ 2581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:59.071391 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:59.071395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:59.071397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:59.071517 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:59.071900 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:59.071910 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:59.071912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:59.072180 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:59.072295 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:59.073180 (+ 885us) server_negotiation.cc:299] Negotiation successful
0926 15:02:59.073408 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 15:02:59.074775 14308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:59.056208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:42813 (local address 127.9.123.67:47021)
0926 15:02:59.057015 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:59.057067 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:59.059413 (+ 2346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:59.060286 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:59.060299 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:59.061013 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:02:59.061691 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:59.061710 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.062835 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.062840 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:59.064118 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:59.064131 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.064821 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.064826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:59.065018 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:59.065644 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:02:59.065665 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:02:59.067786 (+ 2121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:02:59.070796 (+ 3010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:59.070806 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:59.070823 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:59.071265 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:59.071610 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:02:59.071614 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:02:59.071617 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:02:59.071809 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:02:59.072354 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:02:59.072360 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:02:59.072709 (+ 349us) client_negotiation.cc:769] Sending connection context
0926 15:02:59.072964 (+ 255us) client_negotiation.cc:240] Negotiation successful
0926 15:02:59.073443 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":296,"threads_started":1}
I20260926 15:02:59.076203 14306 heartbeater.cc:347] Connected to a master server at 127.9.123.126:42813
I20260926 15:02:59.076565 14306 heartbeater.cc:464] Registering TS with master...
I20260926 15:02:59.077363 14306 heartbeater.cc:511] Master 127.9.123.126:42813 requested a full tablet report, sending...
I20260926 15:02:59.078868 13853 ts_manager.cc:194] Registered new tserver with Master: bd29214bcdd04a4f90b7f18642d74dec (127.9.123.67:36587)
I20260926 15:02:59.079682 13853 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.67@KRBTEST.COM'} at 127.9.123.67:47021
I20260926 15:02:59.088279 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 15:02:59.090029 14041 heartbeater.cc:503] Master 127.9.123.126:42813 was elected leader, sending a full tablet report...
I20260926 15:02:59.105504 14043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:59.093254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48468 (local address 127.9.123.126:42813)
0926 15:02:59.093417 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:59.093423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:59.093926 (+ 503us) server_negotiation.cc:407] Connection header received
0926 15:02:59.094094 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:59.094099 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:59.094160 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:59.094237 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:59.095588 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.096308 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.097587 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.098017 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.098128 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:59.100363 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:59.100368 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:59.100371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:59.100405 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:59.102935 (+ 2530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:59.103551 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:59.103557 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:59.103560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:59.103703 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:59.104049 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:59.104053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:59.104056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:59.104390 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:59.104617 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:59.104946 (+ 329us) server_negotiation.cc:299] Negotiation successful
0926 15:02:59.105188 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 15:02:59.109817 13853 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:02:59.116772 13853 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:02:59.140199 14319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:59.130289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46366 (local address 127.9.123.66:45047)
0926 15:02:59.132787 (+ 2498us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:59.132795 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:59.132840 (+ 45us) server_negotiation.cc:407] Connection header received
0926 15:02:59.132942 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:59.132950 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:59.133267 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:59.134267 (+ 1000us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:59.134497 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.135816 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.138146 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.139059 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.139180 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:59.139385 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:59.139680 (+ 295us) server_negotiation.cc:299] Negotiation successful
0926 15:02:59.139908 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2364,"thread_start_us":2265,"threads_started":1}
I20260926 15:02:59.140199 14318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:59.130110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:45047 (local address 127.0.0.1:46366)
0926 15:02:59.130834 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:59.130878 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:59.131066 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:59.133508 (+ 2442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:59.133512 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:59.133544 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:59.134082 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:59.134094 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.135940 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.135949 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:59.138013 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:59.138024 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.139249 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.139256 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:59.139440 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:59.139495 (+ 55us) client_negotiation.cc:769] Sending connection context
0926 15:02:59.139608 (+ 113us) client_negotiation.cc:240] Negotiation successful
0926 15:02:59.139795 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":319,"threads_started":1}
I20260926 15:02:59.141336 14320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:59.131083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:34153 (local address 127.0.0.1:51972)
0926 15:02:59.131608 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:59.131630 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:59.131743 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:59.133123 (+ 1380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:59.133133 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:59.133173 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:59.133803 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:59.133817 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.135309 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.135314 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:59.137852 (+ 2538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:59.137865 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.140022 (+ 2157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.140028 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:59.140206 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:59.140239 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 15:02:59.140320 (+ 81us) client_negotiation.cc:240] Negotiation successful
0926 15:02:59.140981 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":851,"threads_started":1}
I20260926 15:02:59.142292 14322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:59.131729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51972 (local address 127.9.123.65:34153)
0926 15:02:59.132378 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:59.132389 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:59.132422 (+ 33us) server_negotiation.cc:407] Connection header received
0926 15:02:59.132532 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:59.132542 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:59.132846 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:59.133708 (+ 862us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:59.133942 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.135170 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.138023 (+ 2853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.139528 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.139662 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:59.139883 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:59.141792 (+ 1909us) server_negotiation.cc:299] Negotiation successful
0926 15:02:59.141980 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":379,"threads_started":1}
I20260926 15:02:59.142321 14321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:59.131251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:36587 (local address 127.0.0.1:41020)
0926 15:02:59.134428 (+ 3177us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:59.134455 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:59.134563 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:59.135246 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:59.135251 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:59.135277 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:59.135705 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:59.135716 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.137177 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.137182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:59.139590 (+ 2408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:59.139603 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.141621 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.141628 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:59.141790 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:59.141823 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 15:02:59.141912 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 15:02:59.142026 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2308,"thread_start_us":2130,"threads_started":1}
I20260926 15:02:59.142442 14323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:59.131846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41020 (local address 127.9.123.67:36587)
0926 15:02:59.133229 (+ 1383us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:59.133236 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:59.134681 (+ 1445us) server_negotiation.cc:407] Connection header received
0926 15:02:59.134780 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:59.134788 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:59.135051 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:59.135233 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:59.135897 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.137015 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.139754 (+ 2739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.141145 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.141258 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:59.141486 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:59.141973 (+ 487us) server_negotiation.cc:299] Negotiation successful
0926 15:02:59.142129 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1199,"thread_start_us":1050,"threads_started":1}
I20260926 15:02:59.144627 14109 tablet_service.cc:1467] Processing CreateTablet for tablet e56e993b72c14497a61f772a5a53c501 (DEFAULT_TABLE table=test-table [id=2cef244dc24c49e08d8d2fad8ee64c60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:59.145614 13976 tablet_service.cc:1467] Processing CreateTablet for tablet e56e993b72c14497a61f772a5a53c501 (DEFAULT_TABLE table=test-table [id=2cef244dc24c49e08d8d2fad8ee64c60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:59.145872 14241 tablet_service.cc:1467] Processing CreateTablet for tablet e56e993b72c14497a61f772a5a53c501 (DEFAULT_TABLE table=test-table [id=2cef244dc24c49e08d8d2fad8ee64c60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:02:59.146673 14109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e56e993b72c14497a61f772a5a53c501. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:59.146931 13976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e56e993b72c14497a61f772a5a53c501. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:59.147133 14241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e56e993b72c14497a61f772a5a53c501. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:02:59.158614 14324 tablet_bootstrap.cc:492] T e56e993b72c14497a61f772a5a53c501 P ef992cab50374dc9a4c59e59109d2d95: Bootstrap starting.
I20260926 15:02:59.160949 14325 tablet_bootstrap.cc:492] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0: Bootstrap starting.
I20260926 15:02:59.161748 14326 tablet_bootstrap.cc:492] T e56e993b72c14497a61f772a5a53c501 P bd29214bcdd04a4f90b7f18642d74dec: Bootstrap starting.
I20260926 15:02:59.162384 14324 tablet_bootstrap.cc:654] T e56e993b72c14497a61f772a5a53c501 P ef992cab50374dc9a4c59e59109d2d95: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:59.163744 14324 log.cc:824] T e56e993b72c14497a61f772a5a53c501 P ef992cab50374dc9a4c59e59109d2d95: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:59.165467 14326 tablet_bootstrap.cc:654] T e56e993b72c14497a61f772a5a53c501 P bd29214bcdd04a4f90b7f18642d74dec: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:59.167889 14326 log.cc:824] T e56e993b72c14497a61f772a5a53c501 P bd29214bcdd04a4f90b7f18642d74dec: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:59.168587 14325 tablet_bootstrap.cc:654] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0: Neither blocks nor log segments found. Creating new log.
I20260926 15:02:59.169775 14324 tablet_bootstrap.cc:492] T e56e993b72c14497a61f772a5a53c501 P ef992cab50374dc9a4c59e59109d2d95: No bootstrap required, opened a new log
I20260926 15:02:59.169796 14325 log.cc:824] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0: Log is configured to *not* fsync() on all Append() calls
I20260926 15:02:59.170053 14324 ts_tablet_manager.cc:1397] T e56e993b72c14497a61f772a5a53c501 P ef992cab50374dc9a4c59e59109d2d95: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.005s
I20260926 15:02:59.170907 14326 tablet_bootstrap.cc:492] T e56e993b72c14497a61f772a5a53c501 P bd29214bcdd04a4f90b7f18642d74dec: No bootstrap required, opened a new log
I20260926 15:02:59.171144 14326 ts_tablet_manager.cc:1397] T e56e993b72c14497a61f772a5a53c501 P bd29214bcdd04a4f90b7f18642d74dec: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260926 15:02:59.172703 14325 tablet_bootstrap.cc:492] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0: No bootstrap required, opened a new log
I20260926 15:02:59.173015 14325 ts_tablet_manager.cc:1397] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260926 15:02:59.178911 14326 raft_consensus.cc:348] T e56e993b72c14497a61f772a5a53c501 P bd29214bcdd04a4f90b7f18642d74dec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3ff5fbba214909b1a8040e252021d0" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45047 } } peers { permanent_uuid: "bd29214bcdd04a4f90b7f18642d74dec" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 36587 } } peers { permanent_uuid: "ef992cab50374dc9a4c59e59109d2d95" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34153 } }
I20260926 15:02:59.179340 14326 raft_consensus.cc:374] T e56e993b72c14497a61f772a5a53c501 P bd29214bcdd04a4f90b7f18642d74dec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:59.179466 14326 raft_consensus.cc:729] T e56e993b72c14497a61f772a5a53c501 P bd29214bcdd04a4f90b7f18642d74dec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd29214bcdd04a4f90b7f18642d74dec, State: Initialized, Role: FOLLOWER
I20260926 15:02:59.180079 14326 consensus_queue.cc:260] T e56e993b72c14497a61f772a5a53c501 P bd29214bcdd04a4f90b7f18642d74dec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3ff5fbba214909b1a8040e252021d0" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45047 } } peers { permanent_uuid: "bd29214bcdd04a4f90b7f18642d74dec" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 36587 } } peers { permanent_uuid: "ef992cab50374dc9a4c59e59109d2d95" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34153 } }
I20260926 15:02:59.180670 14324 raft_consensus.cc:348] T e56e993b72c14497a61f772a5a53c501 P ef992cab50374dc9a4c59e59109d2d95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3ff5fbba214909b1a8040e252021d0" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45047 } } peers { permanent_uuid: "bd29214bcdd04a4f90b7f18642d74dec" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 36587 } } peers { permanent_uuid: "ef992cab50374dc9a4c59e59109d2d95" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34153 } }
I20260926 15:02:59.181200 14324 raft_consensus.cc:374] T e56e993b72c14497a61f772a5a53c501 P ef992cab50374dc9a4c59e59109d2d95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:59.181202 14306 heartbeater.cc:503] Master 127.9.123.126:42813 was elected leader, sending a full tablet report...
I20260926 15:02:59.181319 14324 raft_consensus.cc:729] T e56e993b72c14497a61f772a5a53c501 P ef992cab50374dc9a4c59e59109d2d95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef992cab50374dc9a4c59e59109d2d95, State: Initialized, Role: FOLLOWER
I20260926 15:02:59.181915 14326 ts_tablet_manager.cc:1428] T e56e993b72c14497a61f772a5a53c501 P bd29214bcdd04a4f90b7f18642d74dec: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260926 15:02:59.182081 14324 consensus_queue.cc:260] T e56e993b72c14497a61f772a5a53c501 P ef992cab50374dc9a4c59e59109d2d95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3ff5fbba214909b1a8040e252021d0" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45047 } } peers { permanent_uuid: "bd29214bcdd04a4f90b7f18642d74dec" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 36587 } } peers { permanent_uuid: "ef992cab50374dc9a4c59e59109d2d95" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34153 } }
I20260926 15:02:59.183521 14325 raft_consensus.cc:348] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3ff5fbba214909b1a8040e252021d0" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45047 } } peers { permanent_uuid: "bd29214bcdd04a4f90b7f18642d74dec" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 36587 } } peers { permanent_uuid: "ef992cab50374dc9a4c59e59109d2d95" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34153 } }
I20260926 15:02:59.183914 14324 ts_tablet_manager.cc:1428] T e56e993b72c14497a61f772a5a53c501 P ef992cab50374dc9a4c59e59109d2d95: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260926 15:02:59.184034 14325 raft_consensus.cc:374] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:02:59.184154 14325 raft_consensus.cc:729] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba3ff5fbba214909b1a8040e252021d0, State: Initialized, Role: FOLLOWER
I20260926 15:02:59.184816 14325 consensus_queue.cc:260] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3ff5fbba214909b1a8040e252021d0" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45047 } } peers { permanent_uuid: "bd29214bcdd04a4f90b7f18642d74dec" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 36587 } } peers { permanent_uuid: "ef992cab50374dc9a4c59e59109d2d95" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34153 } }
I20260926 15:02:59.185961 14174 heartbeater.cc:503] Master 127.9.123.126:42813 was elected leader, sending a full tablet report...
I20260926 15:02:59.186782 14325 ts_tablet_manager.cc:1428] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260926 15:02:59.303666 14175 tablet.cc:2369] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 15:02:59.304504 14307 tablet.cc:2369] T e56e993b72c14497a61f772a5a53c501 P bd29214bcdd04a4f90b7f18642d74dec: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 15:02:59.308386 14042 tablet.cc:2369] T e56e993b72c14497a61f772a5a53c501 P ef992cab50374dc9a4c59e59109d2d95: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:02:59.480443 14332 raft_consensus.cc:482] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:02:59.480751 14332 raft_consensus.cc:504] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3ff5fbba214909b1a8040e252021d0" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45047 } } peers { permanent_uuid: "bd29214bcdd04a4f90b7f18642d74dec" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 36587 } } peers { permanent_uuid: "ef992cab50374dc9a4c59e59109d2d95" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34153 } }
I20260926 15:02:59.482509 14332 leader_election.cc:288] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd29214bcdd04a4f90b7f18642d74dec (127.9.123.67:36587), ef992cab50374dc9a4c59e59109d2d95 (127.9.123.65:34153)
I20260926 15:02:59.489524 14322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:59.483052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:59543 (local address 127.9.123.65:34153)
0926 15:02:59.483244 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:59.483249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:59.484117 (+ 868us) server_negotiation.cc:407] Connection header received
0926 15:02:59.484270 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:59.484273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:59.484352 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:59.484424 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:59.485039 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.485901 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.487847 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.488608 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.488754 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:59.488831 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:59.489142 (+ 311us) server_negotiation.cc:299] Negotiation successful
0926 15:02:59.489264 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 15:02:59.489641 14323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:59.482888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:53775 (local address 127.9.123.67:36587)
0926 15:02:59.483095 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:59.483101 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:59.484045 (+ 944us) server_negotiation.cc:407] Connection header received
0926 15:02:59.484233 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:59.484239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:59.484300 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:59.484378 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:02:59.484983 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.485899 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.487942 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.488719 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.488870 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:59.488942 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:59.489261 (+ 319us) server_negotiation.cc:299] Negotiation successful
0926 15:02:59.489396 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 15:02:59.489673 14334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:59.483181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:34153 (local address 127.9.123.66:59543)
0926 15:02:59.484014 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:59.484029 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:59.484150 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:59.484449 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:59.484453 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:59.484475 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:59.484898 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:59.484907 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.486021 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.486025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:59.487688 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:59.487701 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.488818 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.488824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:59.488978 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:59.489002 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 15:02:59.489080 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 15:02:59.489231 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":84,"thread_start_us":407,"threads_started":1}
I20260926 15:02:59.490005 14333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:59.483146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:36587 (local address 127.9.123.66:53775)
0926 15:02:59.483931 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0926 15:02:59.483951 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 15:02:59.484102 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:02:59.484406 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:02:59.484410 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:02:59.484433 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:02:59.484837 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:59.484851 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.486007 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.486012 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:59.487788 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:02:59.487800 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.488937 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.488944 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:02:59.489089 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:59.489146 (+ 57us) client_negotiation.cc:769] Sending connection context
0926 15:02:59.489230 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 15:02:59.489357 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"mutex_wait_us":65,"thread_start_us":380,"threads_started":1}
I20260926 15:02:59.490972 14261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e56e993b72c14497a61f772a5a53c501" candidate_uuid: "ba3ff5fbba214909b1a8040e252021d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd29214bcdd04a4f90b7f18642d74dec" is_pre_election: true
I20260926 15:02:59.490972 13996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e56e993b72c14497a61f772a5a53c501" candidate_uuid: "ba3ff5fbba214909b1a8040e252021d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef992cab50374dc9a4c59e59109d2d95" is_pre_election: true
I20260926 15:02:59.491534 14261 raft_consensus.cc:2454] T e56e993b72c14497a61f772a5a53c501 P bd29214bcdd04a4f90b7f18642d74dec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba3ff5fbba214909b1a8040e252021d0 in term 0.
I20260926 15:02:59.491533 13996 raft_consensus.cc:2454] T e56e993b72c14497a61f772a5a53c501 P ef992cab50374dc9a4c59e59109d2d95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba3ff5fbba214909b1a8040e252021d0 in term 0.
I20260926 15:02:59.492292 14062 leader_election.cc:302] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba3ff5fbba214909b1a8040e252021d0, bd29214bcdd04a4f90b7f18642d74dec; no voters:
I20260926 15:02:59.492702 14332 raft_consensus.cc:2781] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:02:59.492878 14332 raft_consensus.cc:482] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:02:59.492990 14332 raft_consensus.cc:3037] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:59.495133 14332 raft_consensus.cc:504] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3ff5fbba214909b1a8040e252021d0" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45047 } } peers { permanent_uuid: "bd29214bcdd04a4f90b7f18642d74dec" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 36587 } } peers { permanent_uuid: "ef992cab50374dc9a4c59e59109d2d95" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34153 } }
I20260926 15:02:59.495723 14332 leader_election.cc:288] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0 [CANDIDATE]: Term 1 election: Requested vote from peers bd29214bcdd04a4f90b7f18642d74dec (127.9.123.67:36587), ef992cab50374dc9a4c59e59109d2d95 (127.9.123.65:34153)
I20260926 15:02:59.496320 14261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e56e993b72c14497a61f772a5a53c501" candidate_uuid: "ba3ff5fbba214909b1a8040e252021d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd29214bcdd04a4f90b7f18642d74dec"
I20260926 15:02:59.496312 13996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e56e993b72c14497a61f772a5a53c501" candidate_uuid: "ba3ff5fbba214909b1a8040e252021d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef992cab50374dc9a4c59e59109d2d95"
I20260926 15:02:59.496546 13996 raft_consensus.cc:3037] T e56e993b72c14497a61f772a5a53c501 P ef992cab50374dc9a4c59e59109d2d95 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:59.496544 14261 raft_consensus.cc:3037] T e56e993b72c14497a61f772a5a53c501 P bd29214bcdd04a4f90b7f18642d74dec [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:02:59.499796 14261 raft_consensus.cc:2454] T e56e993b72c14497a61f772a5a53c501 P bd29214bcdd04a4f90b7f18642d74dec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba3ff5fbba214909b1a8040e252021d0 in term 1.
I20260926 15:02:59.499796 13996 raft_consensus.cc:2454] T e56e993b72c14497a61f772a5a53c501 P ef992cab50374dc9a4c59e59109d2d95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba3ff5fbba214909b1a8040e252021d0 in term 1.
I20260926 15:02:59.500285 14063 leader_election.cc:302] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba3ff5fbba214909b1a8040e252021d0, ef992cab50374dc9a4c59e59109d2d95; no voters:
I20260926 15:02:59.500660 14332 raft_consensus.cc:2781] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:02:59.501127 14332 raft_consensus.cc:686] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0 [term 1 LEADER]: Becoming Leader. State: Replica: ba3ff5fbba214909b1a8040e252021d0, State: Running, Role: LEADER
I20260926 15:02:59.501680 14332 consensus_queue.cc:237] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3ff5fbba214909b1a8040e252021d0" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45047 } } peers { permanent_uuid: "bd29214bcdd04a4f90b7f18642d74dec" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 36587 } } peers { permanent_uuid: "ef992cab50374dc9a4c59e59109d2d95" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34153 } }
I20260926 15:02:59.506304 13853 catalog_manager.cc:5495] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba3ff5fbba214909b1a8040e252021d0 (127.9.123.66). New cstate: current_term: 1 leader_uuid: "ba3ff5fbba214909b1a8040e252021d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3ff5fbba214909b1a8040e252021d0" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd29214bcdd04a4f90b7f18642d74dec" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 36587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef992cab50374dc9a4c59e59109d2d95" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34153 } health_report { overall_health: UNKNOWN } } }
I20260926 15:02:59.601470 14319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:59.595830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40842 (local address 127.9.123.66:45047)
0926 15:02:59.596049 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:59.596055 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:59.596147 (+ 92us) server_negotiation.cc:407] Connection header received
0926 15:02:59.596386 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:59.596390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:59.596467 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:59.596599 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:02:59.596987 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.597724 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.599143 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.599578 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.599706 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:59.600197 (+ 491us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:02:59.600789 (+ 592us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:02:59.600932 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:59.601057 (+ 125us) server_negotiation.cc:299] Negotiation successful
0926 15:02:59.601189 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 15:02:59.609769 14261 raft_consensus.cc:1261] T e56e993b72c14497a61f772a5a53c501 P bd29214bcdd04a4f90b7f18642d74dec [term 1 FOLLOWER]: Refusing update from remote peer ba3ff5fbba214909b1a8040e252021d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:02:59.609769 13996 raft_consensus.cc:1261] T e56e993b72c14497a61f772a5a53c501 P ef992cab50374dc9a4c59e59109d2d95 [term 1 FOLLOWER]: Refusing update from remote peer ba3ff5fbba214909b1a8040e252021d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:02:59.610849 14335 consensus_queue.cc:1035] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd29214bcdd04a4f90b7f18642d74dec" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 36587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:59.611210 14332 consensus_queue.cc:1035] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef992cab50374dc9a4c59e59109d2d95" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:02:59.636307 13853 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:02:59.636540 13853 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:02:59.640013 13853 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fd71bca671bb49a4a5fbd031480cfd67: Sending DeleteTablet for 3 replicas of tablet e56e993b72c14497a61f772a5a53c501
I20260926 15:02:59.641258 14109 tablet_service.cc:1514] Processing DeleteTablet for tablet e56e993b72c14497a61f772a5a53c501 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:02:59 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:46366
I20260926 15:02:59.641278 13976 tablet_service.cc:1514] Processing DeleteTablet for tablet e56e993b72c14497a61f772a5a53c501 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:02:59 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:51972
I20260926 15:02:59.641937 14241 tablet_service.cc:1514] Processing DeleteTablet for tablet e56e993b72c14497a61f772a5a53c501 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:02:59 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:41020
I20260926 15:02:59.642982 14348 tablet_replica.cc:330] T e56e993b72c14497a61f772a5a53c501 P ef992cab50374dc9a4c59e59109d2d95: stopping tablet replica
I20260926 15:02:59.643230 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 13914
I20260926 15:02:59.643505 14348 raft_consensus.cc:2229] T e56e993b72c14497a61f772a5a53c501 P ef992cab50374dc9a4c59e59109d2d95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:59.644032 14348 raft_consensus.cc:2258] T e56e993b72c14497a61f772a5a53c501 P ef992cab50374dc9a4c59e59109d2d95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:59.650708 14348 ts_tablet_manager.cc:1905] T e56e993b72c14497a61f772a5a53c501 P ef992cab50374dc9a4c59e59109d2d95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:59.655978 14350 tablet_replica.cc:330] T e56e993b72c14497a61f772a5a53c501 P bd29214bcdd04a4f90b7f18642d74dec: stopping tablet replica
I20260926 15:02:59.658568 14350 raft_consensus.cc:2229] T e56e993b72c14497a61f772a5a53c501 P bd29214bcdd04a4f90b7f18642d74dec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:02:59.659245 14350 raft_consensus.cc:2258] T e56e993b72c14497a61f772a5a53c501 P bd29214bcdd04a4f90b7f18642d74dec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:59.661007 14348 ts_tablet_manager.cc:1918] T e56e993b72c14497a61f772a5a53c501 P ef992cab50374dc9a4c59e59109d2d95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:02:59.661257 14348 log.cc:1196] T e56e993b72c14497a61f772a5a53c501 P ef992cab50374dc9a4c59e59109d2d95: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-0/wal/wals/e56e993b72c14497a61f772a5a53c501
I20260926 15:02:59.661602 14349 tablet_replica.cc:330] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0: stopping tablet replica
I20260926 15:02:59.661877 14348 ts_tablet_manager.cc:1939] T e56e993b72c14497a61f772a5a53c501 P ef992cab50374dc9a4c59e59109d2d95: Deleting consensus metadata
I20260926 15:02:59.662101 14349 raft_consensus.cc:2229] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0 [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:02:59.662828 14349 raft_consensus.cc:2258] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:02:59.663969 14350 ts_tablet_manager.cc:1905] T e56e993b72c14497a61f772a5a53c501 P bd29214bcdd04a4f90b7f18642d74dec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:02:59.664366 13838 catalog_manager.cc:4841] TS ef992cab50374dc9a4c59e59109d2d95 (127.9.123.65:34153): tablet e56e993b72c14497a61f772a5a53c501 (table test-table [id=2cef244dc24c49e08d8d2fad8ee64c60]) successfully deleted
I20260926 15:02:59.664734 14349 ts_tablet_manager.cc:1905] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 26 15:02:59 dist-test-slave-7153 krb5kdc[13809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.65@KRBTEST.COM
I20260926 15:02:59.671849 14350 ts_tablet_manager.cc:1918] T e56e993b72c14497a61f772a5a53c501 P bd29214bcdd04a4f90b7f18642d74dec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:02:59.672130 14350 log.cc:1196] T e56e993b72c14497a61f772a5a53c501 P bd29214bcdd04a4f90b7f18642d74dec: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/e56e993b72c14497a61f772a5a53c501
I20260926 15:02:59.672698 14350 ts_tablet_manager.cc:1939] T e56e993b72c14497a61f772a5a53c501 P bd29214bcdd04a4f90b7f18642d74dec: Deleting consensus metadata
I20260926 15:02:59.672773 14349 ts_tablet_manager.cc:1918] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:02:59.672993 14349 log.cc:1196] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/e56e993b72c14497a61f772a5a53c501
I20260926 15:02:59.673611 14349 ts_tablet_manager.cc:1939] T e56e993b72c14497a61f772a5a53c501 P ba3ff5fbba214909b1a8040e252021d0: Deleting consensus metadata
I20260926 15:02:59.674832 13837 catalog_manager.cc:4841] TS bd29214bcdd04a4f90b7f18642d74dec (127.9.123.67:36587): tablet e56e993b72c14497a61f772a5a53c501 (table test-table [id=2cef244dc24c49e08d8d2fad8ee64c60]) successfully deleted
I20260926 15:02:59.675699 13837 catalog_manager.cc:4841] TS ba3ff5fbba214909b1a8040e252021d0 (127.9.123.66:45047): tablet e56e993b72c14497a61f772a5a53c501 (table test-table [id=2cef244dc24c49e08d8d2fad8ee64c60]) successfully deleted
I20260926 15:02:59.679282 14322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:59.646224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33464 (local address 127.9.123.65:34153)
0926 15:02:59.646410 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:59.646428 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:59.655041 (+ 8613us) server_negotiation.cc:407] Connection header received
0926 15:02:59.656256 (+ 1215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:59.656262 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:59.656351 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:59.656436 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:59.659773 (+ 3337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.660841 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.664651 (+ 3810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.665179 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.665261 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:59.672678 (+ 7417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:59.672699 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:59.672718 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:59.672771 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:59.676242 (+ 3471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:59.677019 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:59.677031 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:59.677035 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:59.677243 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:59.677592 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:59.677597 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:59.677601 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:59.678077 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:59.678373 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:59.678694 (+ 321us) server_negotiation.cc:299] Negotiation successful
0926 15:02:59.678914 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 15:02:59.679970 14036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:02:59.767488 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 13914
I20260926 15:02:59.781213 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 14047
Sep 26 15:02:59 dist-test-slave-7153 krb5kdc[13809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.66@KRBTEST.COM
I20260926 15:02:59.799085 14319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:59.781962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40856 (local address 127.9.123.66:45047)
0926 15:02:59.782150 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:59.782156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:59.782178 (+ 22us) server_negotiation.cc:407] Connection header received
0926 15:02:59.782296 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:59.782302 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:59.782437 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:59.782569 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:59.783652 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.784855 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.786246 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.786985 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.787115 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:59.791579 (+ 4464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:59.791601 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:59.791628 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:59.791688 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:59.795706 (+ 4018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:59.796523 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:59.796533 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:59.796537 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:59.796692 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:59.797087 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:59.797093 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:59.797171 (+ 78us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:59.797682 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:59.798085 (+ 403us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:59.798417 (+ 332us) server_negotiation.cc:299] Negotiation successful
0926 15:02:59.798687 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 15:02:59.799975 14169 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:02:59.911587 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 14047
I20260926 15:02:59.925917 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 14179
Sep 26 15:02:59 dist-test-slave-7153 krb5kdc[13809](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.67@KRBTEST.COM
I20260926 15:02:59.940320 14323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:02:59.926659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46466 (local address 127.9.123.67:36587)
0926 15:02:59.926848 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 15:02:59.926852 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:02:59.926871 (+ 19us) server_negotiation.cc:407] Connection header received
0926 15:02:59.926983 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:02:59.926987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:02:59.927047 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:02:59.927131 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:02:59.928313 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.929048 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.930298 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:02:59.930744 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:02:59.930856 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:02:59.935063 (+ 4207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:02:59.935077 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:02:59.935093 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:02:59.935134 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:02:59.937780 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:59.938430 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:59.938437 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:59.938439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:59.938593 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:02:59.939021 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:02:59.939025 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:02:59.939027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:02:59.939399 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:02:59.939605 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0926 15:02:59.939879 (+ 274us) server_negotiation.cc:299] Negotiation successful
0926 15:02:59.940035 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 15:02:59.940932 14301 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:00.027592 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 14179
I20260926 15:03:00.040248 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 13822
I20260926 15:03:00.041429 13883 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:00.147472 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 13822
2026-09-26T15:03:00Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4344 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-26T15:03:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:03:00Z Disabled control of system clock
I20260926 15:03:00.207216 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:42229
--webserver_interface=127.9.123.126
--webserver_port=0
--builtin_ntp_servers=127.9.123.84:36765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:42229
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 15:03:00.534813 14359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:00.535133 14359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:00.535219 14359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:00.542523 14359 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:03:00.542644 14359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:00.542678 14359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:00.542737 14359 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:03:00.542779 14359 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:03:00.552210 14359 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:36765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:42229
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:42229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:00.553527 14359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:00.555337 14359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:00.567008 14364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:00.567050 14367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:00.567082 14365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:00.567884 14359 server_base.cc:1034] running on GCE node
I20260926 15:03:00.568722 14359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:00.570302 14359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:00.571512 14359 hybrid_clock.cc:648] HybridClock initialized: now 1790434980571467 us; error 44 us; skew 500 ppm
I20260926 15:03:00.571872 14359 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:03:00.574213 14359 webserver.cc:460] Webserver started at http://127.9.123.126:41059/ using document root <none> and password file <none>
I20260926 15:03:00.574930 14359 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:00.575034 14359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:00.575340 14359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:00.578334 14359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "ea181dc3ec4847f6b265725049e891e1"
format_stamp: "Formatted at 2026-09-26 15:03:00 on dist-test-slave-7153"
server_key: "41fe0e04e50dfb1e9e807cca5b26ef89"
server_key_iv: "83b5995dbcedae56f418cb6266b4ce11"
server_key_version: "encryptionkey@0"
I20260926 15:03:00.579136 14359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "ea181dc3ec4847f6b265725049e891e1"
format_stamp: "Formatted at 2026-09-26 15:03:00 on dist-test-slave-7153"
server_key: "41fe0e04e50dfb1e9e807cca5b26ef89"
server_key_iv: "83b5995dbcedae56f418cb6266b4ce11"
server_key_version: "encryptionkey@0"
I20260926 15:03:00.584537 14359 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 15:03:00.588263 14373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:00.589797 14359 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 15:03:00.589982 14359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "ea181dc3ec4847f6b265725049e891e1"
format_stamp: "Formatted at 2026-09-26 15:03:00 on dist-test-slave-7153"
server_key: "41fe0e04e50dfb1e9e807cca5b26ef89"
server_key_iv: "83b5995dbcedae56f418cb6266b4ce11"
server_key_version: "encryptionkey@0"
I20260926 15:03:00.590194 14359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:00.596266 14359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:00.597155 14359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:00.597443 14359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:00.614794 14359 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:42229
I20260926 15:03:00.614827 14424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:42229 every 8 connection(s)
I20260926 15:03:00.616123 14359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:03:00.619830 14425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:00.626057 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 14359
I20260926 15:03:00.626236 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:03:00.626870 9709 external_mini_cluster.cc:1428] Setting key 6bd4242ecf27d134b4aa56e0710cc5a3
I20260926 15:03:00.631172 14425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea181dc3ec4847f6b265725049e891e1: Bootstrap starting.
I20260926 15:03:00.635408 14425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea181dc3ec4847f6b265725049e891e1: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:00.636737 14428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:00.628989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46256 (local address 127.9.123.126:42229)
0926 15:03:00.630133 (+ 1144us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:00.630148 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:00.630203 (+ 55us) server_negotiation.cc:407] Connection header received
0926 15:03:00.632461 (+ 2258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:00.632492 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:00.633155 (+ 663us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:00.633659 (+ 504us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:00.634686 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:00.634694 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:00.634711 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:00.634728 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:00.634807 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:00.634829 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:00.635029 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:00.635157 (+ 128us) server_negotiation.cc:299] Negotiation successful
0926 15:03:00.635459 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":311,"threads_started":1}
I20260926 15:03:00.636823 14425 log.cc:824] T 00000000000000000000000000000000 P ea181dc3ec4847f6b265725049e891e1: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:00.640054 14425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea181dc3ec4847f6b265725049e891e1: No bootstrap required, opened a new log
I20260926 15:03:00.648248 14425 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea181dc3ec4847f6b265725049e891e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea181dc3ec4847f6b265725049e891e1" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42229 } }
I20260926 15:03:00.648792 14425 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea181dc3ec4847f6b265725049e891e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:00.648918 14425 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea181dc3ec4847f6b265725049e891e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea181dc3ec4847f6b265725049e891e1, State: Initialized, Role: FOLLOWER
I20260926 15:03:00.649703 14425 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea181dc3ec4847f6b265725049e891e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea181dc3ec4847f6b265725049e891e1" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42229 } }
I20260926 15:03:00.649989 14425 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea181dc3ec4847f6b265725049e891e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:03:00.650090 14425 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea181dc3ec4847f6b265725049e891e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:03:00.650226 14425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea181dc3ec4847f6b265725049e891e1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:00.652433 14425 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea181dc3ec4847f6b265725049e891e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea181dc3ec4847f6b265725049e891e1" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42229 } }
I20260926 15:03:00.652961 14425 leader_election.cc:302] T 00000000000000000000000000000000 P ea181dc3ec4847f6b265725049e891e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea181dc3ec4847f6b265725049e891e1; no voters:
I20260926 15:03:00.653640 14425 leader_election.cc:288] T 00000000000000000000000000000000 P ea181dc3ec4847f6b265725049e891e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:03:00.653769 14430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea181dc3ec4847f6b265725049e891e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:03:00.654436 14430 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea181dc3ec4847f6b265725049e891e1 [term 1 LEADER]: Becoming Leader. State: Replica: ea181dc3ec4847f6b265725049e891e1, State: Running, Role: LEADER
I20260926 15:03:00.655040 14430 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea181dc3ec4847f6b265725049e891e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea181dc3ec4847f6b265725049e891e1" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42229 } }
I20260926 15:03:00.655931 14425 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea181dc3ec4847f6b265725049e891e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:03:00.657800 14432 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea181dc3ec4847f6b265725049e891e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea181dc3ec4847f6b265725049e891e1. Latest consensus state: current_term: 1 leader_uuid: "ea181dc3ec4847f6b265725049e891e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea181dc3ec4847f6b265725049e891e1" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42229 } } }
I20260926 15:03:00.658192 14432 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea181dc3ec4847f6b265725049e891e1 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:00.658088 14431 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea181dc3ec4847f6b265725049e891e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea181dc3ec4847f6b265725049e891e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea181dc3ec4847f6b265725049e891e1" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42229 } } }
I20260926 15:03:00.658388 14431 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea181dc3ec4847f6b265725049e891e1 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:00.659898 14438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:03:00.664984 14438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:03:00.674556 14438 catalog_manager.cc:1347] Generated new cluster ID: b709edb91141498391b4281071130de4
I20260926 15:03:00.674675 14438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:03:00.698565 14438 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:03:00.699021 14438 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:03:00.713572 14438 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea181dc3ec4847f6b265725049e891e1: Generated new TSK 0
I20260926 15:03:00.714509 14438 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:03:00.724644 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:42229
--builtin_ntp_servers=127.9.123.84:36765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260926 15:03:01.073339 14449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:01.073671 14449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:01.073782 14449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:01.081159 14449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:01.081280 14449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:01.081415 14449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:03:01.090931 14449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:36765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:42229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:01.092278 14449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:01.094152 14449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:01.107242 14454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:01.107209 14457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:01.107280 14455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:01.108233 14449 server_base.cc:1034] running on GCE node
I20260926 15:03:01.108862 14449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:01.109951 14449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:01.111163 14449 hybrid_clock.cc:648] HybridClock initialized: now 1790434981111112 us; error 44 us; skew 500 ppm
I20260926 15:03:01.111472 14449 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:03:01.113790 14449 webserver.cc:460] Webserver started at http://127.9.123.65:38953/ using document root <none> and password file <none>
I20260926 15:03:01.114480 14449 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:01.114581 14449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:01.114876 14449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:01.117836 14449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "225f7bc90df54d4297d8d0adcb01d1c6"
format_stamp: "Formatted at 2026-09-26 15:03:01 on dist-test-slave-7153"
server_key: "5929cc02db3fae9d51a1f87a3f10936d"
server_key_iv: "358eb7405eac46bd2a162adc54952f85"
server_key_version: "encryptionkey@0"
I20260926 15:03:01.118634 14449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "225f7bc90df54d4297d8d0adcb01d1c6"
format_stamp: "Formatted at 2026-09-26 15:03:01 on dist-test-slave-7153"
server_key: "5929cc02db3fae9d51a1f87a3f10936d"
server_key_iv: "358eb7405eac46bd2a162adc54952f85"
server_key_version: "encryptionkey@0"
I20260926 15:03:01.124099 14449 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 15:03:01.128191 14463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:01.129711 14449 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:03:01.129868 14449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "225f7bc90df54d4297d8d0adcb01d1c6"
format_stamp: "Formatted at 2026-09-26 15:03:01 on dist-test-slave-7153"
server_key: "5929cc02db3fae9d51a1f87a3f10936d"
server_key_iv: "358eb7405eac46bd2a162adc54952f85"
server_key_version: "encryptionkey@0"
I20260926 15:03:01.130108 14449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:01.136217 14449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:01.137087 14449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:01.137377 14449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:01.138291 14449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:01.139681 14449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:01.139773 14449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:01.139878 14449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:01.139942 14449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:01.168905 14449 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:34299
I20260926 15:03:01.168958 14575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:34299 every 8 connection(s)
I20260926 15:03:01.170208 14449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:03:01.176419 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 14449
I20260926 15:03:01.176597 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:03:01.177356 9709 external_mini_cluster.cc:1428] Setting key 7303e628f11584b77b8bd250153ab947
I20260926 15:03:01.178252 14578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:01.172686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:58063 (local address 127.9.123.126:42229)
0926 15:03:01.173091 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:01.173119 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:01.174245 (+ 1126us) server_negotiation.cc:407] Connection header received
0926 15:03:01.175905 (+ 1660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:01.175911 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:01.175984 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:01.176078 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:01.177519 (+ 1441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:01.177523 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:01.177527 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:01.177544 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:01.177589 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:01.177601 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:01.177709 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:01.177898 (+ 189us) server_negotiation.cc:299] Negotiation successful
0926 15:03:01.177965 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":210,"threads_started":1}
I20260926 15:03:01.179277 14579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:01.173203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:42229 (local address 127.9.123.65:58063)
0926 15:03:01.174079 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:01.174120 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:01.175629 (+ 1509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:01.176329 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:01.176337 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:01.176447 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:01.177193 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 15:03:01.177196 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:01.177236 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 15:03:01.177335 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:01.177690 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:01.177700 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:01.177708 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 15:03:01.177964 (+ 256us) client_negotiation.cc:240] Negotiation successful
0926 15:03:01.178240 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":239,"threads_started":1}
I20260926 15:03:01.180202 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:42229
--builtin_ntp_servers=127.9.123.84:36765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260926 15:03:01.180783 14576 heartbeater.cc:347] Connected to a master server at 127.9.123.126:42229
I20260926 15:03:01.181128 14576 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:01.181713 14576 heartbeater.cc:511] Master 127.9.123.126:42229 requested a full tablet report, sending...
I20260926 15:03:01.183960 14390 ts_manager.cc:194] Registered new tserver with Master: 225f7bc90df54d4297d8d0adcb01d1c6 (127.9.123.65:34299)
W20260926 15:03:01.516325 14580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:01.516638 14580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:01.516759 14580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:01.525249 14580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:01.525406 14580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:01.525663 14580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:03:01.537757 14580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:36765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:42229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:01.539034 14580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:01.540748 14580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:01.553740 14588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:01.553879 14585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:01.553766 14586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:01.554543 14580 server_base.cc:1034] running on GCE node
I20260926 15:03:01.555215 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:01.556277 14580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:01.557499 14580 hybrid_clock.cc:648] HybridClock initialized: now 1790434981557427 us; error 64 us; skew 500 ppm
I20260926 15:03:01.557859 14580 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:03:01.560534 14580 webserver.cc:460] Webserver started at http://127.9.123.66:44027/ using document root <none> and password file <none>
I20260926 15:03:01.561369 14580 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:01.561470 14580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:01.561776 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:01.564826 14580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "804c8ffefc5f4481b8f74840e8148c3a"
format_stamp: "Formatted at 2026-09-26 15:03:01 on dist-test-slave-7153"
server_key: "60ad2730b57c40a76ea984dcf0ea035a"
server_key_iv: "59a051072e9d48ca66fd7c9e59bf2365"
server_key_version: "encryptionkey@0"
I20260926 15:03:01.565727 14580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "804c8ffefc5f4481b8f74840e8148c3a"
format_stamp: "Formatted at 2026-09-26 15:03:01 on dist-test-slave-7153"
server_key: "60ad2730b57c40a76ea984dcf0ea035a"
server_key_iv: "59a051072e9d48ca66fd7c9e59bf2365"
server_key_version: "encryptionkey@0"
I20260926 15:03:01.571847 14580 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260926 15:03:01.575860 14594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:01.577600 14580 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 15:03:01.577788 14580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "804c8ffefc5f4481b8f74840e8148c3a"
format_stamp: "Formatted at 2026-09-26 15:03:01 on dist-test-slave-7153"
server_key: "60ad2730b57c40a76ea984dcf0ea035a"
server_key_iv: "59a051072e9d48ca66fd7c9e59bf2365"
server_key_version: "encryptionkey@0"
I20260926 15:03:01.578042 14580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:01.584239 14580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:01.585171 14580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:01.585439 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:01.586417 14580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:01.587944 14580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:01.588033 14580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:01.588133 14580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:01.588196 14580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:01.622071 14580 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:36181
I20260926 15:03:01.622135 14706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:36181 every 8 connection(s)
I20260926 15:03:01.623528 14580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:03:01.630779 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 14580
I20260926 15:03:01.630973 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:03:01.631475 9709 external_mini_cluster.cc:1428] Setting key 4a870d1a9f566a8d4483aef6dac02970
I20260926 15:03:01.633387 14578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:01.626241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:51659 (local address 127.9.123.126:42229)
0926 15:03:01.626447 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:01.626454 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:01.628182 (+ 1728us) server_negotiation.cc:407] Connection header received
0926 15:03:01.630707 (+ 2525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:01.630714 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:01.630789 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:01.630979 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:01.632586 (+ 1607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:01.632591 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:01.632596 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:01.632617 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:01.632657 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:01.632671 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:01.632800 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:01.633062 (+ 262us) server_negotiation.cc:299] Negotiation successful
0926 15:03:01.633149 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 15:03:01.634402 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:42229
--builtin_ntp_servers=127.9.123.84:36765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260926 15:03:01.634593 14709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:01.626810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:42229 (local address 127.9.123.66:51659)
0926 15:03:01.627967 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:01.628025 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:01.630327 (+ 2302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:01.631225 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:01.631237 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:01.631376 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:01.632151 (+ 775us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 15:03:01.632154 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:01.632211 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 15:03:01.632333 (+ 122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:01.632800 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:01.632809 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:01.632851 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 15:03:01.633168 (+ 317us) client_negotiation.cc:240] Negotiation successful
0926 15:03:01.633479 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":301,"threads_started":1}
I20260926 15:03:01.636113 14707 heartbeater.cc:347] Connected to a master server at 127.9.123.126:42229
I20260926 15:03:01.636426 14707 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:01.637027 14707 heartbeater.cc:511] Master 127.9.123.126:42229 requested a full tablet report, sending...
I20260926 15:03:01.638443 14390 ts_manager.cc:194] Registered new tserver with Master: 804c8ffefc5f4481b8f74840e8148c3a (127.9.123.66:36181)
W20260926 15:03:01.961408 14710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:01.961710 14710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:01.961836 14710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:01.969213 14710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:01.969336 14710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:01.969496 14710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:03:01.978886 14710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:36765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:42229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:01.980170 14710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:01.981993 14710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:01.994865 14715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:01.994940 14718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:01.994917 14716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:01.995898 14710 server_base.cc:1034] running on GCE node
I20260926 15:03:01.996560 14710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:01.997640 14710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:01.998864 14710 hybrid_clock.cc:648] HybridClock initialized: now 1790434981998787 us; error 73 us; skew 500 ppm
I20260926 15:03:01.999191 14710 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:03:02.001621 14710 webserver.cc:460] Webserver started at http://127.9.123.67:37329/ using document root <none> and password file <none>
I20260926 15:03:02.002322 14710 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:02.002435 14710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:02.002735 14710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:02.005709 14710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "3890a3226c704cbfa68f3b7698b5d93c"
format_stamp: "Formatted at 2026-09-26 15:03:01 on dist-test-slave-7153"
server_key: "fb8709e73db5a2c9b85601b823169b10"
server_key_iv: "17fedb5d741b4f9d2652ac42282d4665"
server_key_version: "encryptionkey@0"
I20260926 15:03:02.006495 14710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "3890a3226c704cbfa68f3b7698b5d93c"
format_stamp: "Formatted at 2026-09-26 15:03:01 on dist-test-slave-7153"
server_key: "fb8709e73db5a2c9b85601b823169b10"
server_key_iv: "17fedb5d741b4f9d2652ac42282d4665"
server_key_version: "encryptionkey@0"
I20260926 15:03:02.011649 14710 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 15:03:02.015178 14724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:02.016604 14710 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 15:03:02.016764 14710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "3890a3226c704cbfa68f3b7698b5d93c"
format_stamp: "Formatted at 2026-09-26 15:03:01 on dist-test-slave-7153"
server_key: "fb8709e73db5a2c9b85601b823169b10"
server_key_iv: "17fedb5d741b4f9d2652ac42282d4665"
server_key_version: "encryptionkey@0"
I20260926 15:03:02.016991 14710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:02.022794 14710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:02.023628 14710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:02.023885 14710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:02.024787 14710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:02.026224 14710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:02.026316 14710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:02.026432 14710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:02.026500 14710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:02.055770 14710 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:34385
I20260926 15:03:02.055800 14836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:34385 every 8 connection(s)
I20260926 15:03:02.057085 14710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:03:02.063128 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 14710
I20260926 15:03:02.063292 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:03:02.063709 9709 external_mini_cluster.cc:1428] Setting key d1ad23cd179f88e3927c2b92093cb13a
I20260926 15:03:02.065362 14578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:02.059545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:58749 (local address 127.9.123.126:42229)
0926 15:03:02.059697 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:02.059701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:02.061333 (+ 1632us) server_negotiation.cc:407] Connection header received
0926 15:03:02.062998 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:02.063004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:02.063081 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:02.063169 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:02.064572 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:02.064578 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:02.064583 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:02.064602 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:02.064641 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:02.064655 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:02.064776 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:02.065019 (+ 243us) server_negotiation.cc:299] Negotiation successful
0926 15:03:02.065075 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 15:03:02.066558 14839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:02.060123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:42229 (local address 127.9.123.67:58749)
0926 15:03:02.061166 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:02.061205 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:02.062710 (+ 1505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:02.063438 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:02.063447 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:02.063548 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:02.064219 (+ 671us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 15:03:02.064222 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:02.064275 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 15:03:02.064392 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:02.064753 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:02.064763 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:02.064771 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 15:03:02.065120 (+ 349us) client_negotiation.cc:240] Negotiation successful
0926 15:03:02.065355 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":291,"threads_started":1}
I20260926 15:03:02.067902 14837 heartbeater.cc:347] Connected to a master server at 127.9.123.126:42229
I20260926 15:03:02.068207 14837 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:02.068787 14837 heartbeater.cc:511] Master 127.9.123.126:42229 requested a full tablet report, sending...
I20260926 15:03:02.070165 14390 ts_manager.cc:194] Registered new tserver with Master: 3890a3226c704cbfa68f3b7698b5d93c (127.9.123.67:34385)
I20260926 15:03:02.079586 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 15:03:02.086565 14578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:02.084080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46266 (local address 127.9.123.126:42229)
0926 15:03:02.084289 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:02.084296 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:02.084643 (+ 347us) server_negotiation.cc:407] Connection header received
0926 15:03:02.084804 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:02.084808 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:02.084862 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:02.084965 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:02.085956 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:02.085961 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:02.085964 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:02.085979 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:02.086009 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:02.086021 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:02.086122 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:02.086248 (+ 126us) server_negotiation.cc:299] Negotiation successful
0926 15:03:02.086298 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 15:03:02.091043 14390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46266:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:03:02.098040 14390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:03:02.153198 14850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:02.110632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:34385 (local address 127.0.0.1:40310)
0926 15:03:02.147767 (+ 37135us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:02.147792 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:02.147938 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:02.150414 (+ 2476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:02.150423 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:02.150542 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:02.151425 (+ 883us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 15:03:02.151429 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:02.151461 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 15:03:02.151543 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:02.152193 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:02.152211 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:02.152220 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 15:03:02.152403 (+ 183us) client_negotiation.cc:240] Negotiation successful
0926 15:03:02.152738 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36980,"thread_start_us":37400,"threads_started":1}
I20260926 15:03:02.153671 14852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:02.110898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40310 (local address 127.9.123.67:34385)
0926 15:03:02.149914 (+ 39016us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:02.149923 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:02.149955 (+ 32us) server_negotiation.cc:407] Connection header received
0926 15:03:02.150019 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:02.150025 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:02.150164 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:02.150313 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:02.151717 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:02.151727 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:02.151744 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:02.151764 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:02.151827 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:02.151849 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:02.153066 (+ 1217us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:02.153161 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 15:03:02.153270 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38839,"thread_start_us":38675,"threads_started":1}
I20260926 15:03:02.153821 14853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:02.110489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:36181 (local address 127.0.0.1:52794)
0926 15:03:02.145392 (+ 34903us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:02.145461 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:02.145659 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:02.152113 (+ 6454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:02.152118 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:02.152141 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:02.152745 (+ 604us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 15:03:02.152747 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:02.152758 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 15:03:02.152794 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:02.153241 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:02.153245 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:02.153250 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 15:03:02.153358 (+ 108us) client_negotiation.cc:240] Negotiation successful
0926 15:03:02.153404 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34517,"thread_start_us":38741,"threads_started":1}
I20260926 15:03:02.154011 14854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:02.112734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46168 (local address 127.9.123.65:34299)
0926 15:03:02.146272 (+ 33538us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:02.146283 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:02.149604 (+ 3321us) server_negotiation.cc:407] Connection header received
0926 15:03:02.149863 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:02.149874 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:02.150159 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:02.150296 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:02.153058 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:02.153066 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:02.153080 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:02.153125 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:02.153187 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:02.153213 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:02.153321 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:02.153537 (+ 216us) server_negotiation.cc:299] Negotiation successful
0926 15:03:02.153642 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33346,"thread_start_us":34377,"threads_started":1}
I20260926 15:03:02.154508 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:02.110735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52794 (local address 127.9.123.66:36181)
0926 15:03:02.151587 (+ 40852us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:02.151595 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:02.151624 (+ 29us) server_negotiation.cc:407] Connection header received
0926 15:03:02.151685 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:02.151692 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:02.151947 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:02.152104 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:02.152963 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:02.152971 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:02.152985 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:02.153010 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:02.153067 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:02.153087 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:02.154052 (+ 965us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:02.154122 (+ 70us) server_negotiation.cc:299] Negotiation successful
0926 15:03:02.154210 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40707,"thread_start_us":40604,"threads_started":1}
I20260926 15:03:02.154666 14849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:02.112187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:34299 (local address 127.0.0.1:46168)
0926 15:03:02.149479 (+ 37292us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:02.149500 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:02.149695 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:02.151249 (+ 1554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:02.151254 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:02.151276 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:02.152615 (+ 1339us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 15:03:02.152618 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:02.152635 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 15:03:02.152894 (+ 259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:02.153403 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:02.153407 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:02.153410 (+ 3us) client_negotiation.cc:769] Sending connection context
0926 15:03:02.154240 (+ 830us) client_negotiation.cc:240] Negotiation successful
0926 15:03:02.154285 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37177,"thread_start_us":34137,"threads_started":1}
I20260926 15:03:02.157943 14642 tablet_service.cc:1467] Processing CreateTablet for tablet dd8fdec0524549a39f607fd9d22b882a (DEFAULT_TABLE table=test-table [id=f9b719b7b7c64ad98ffe2a1dae9c3b3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:03:02.157965 14772 tablet_service.cc:1467] Processing CreateTablet for tablet dd8fdec0524549a39f607fd9d22b882a (DEFAULT_TABLE table=test-table [id=f9b719b7b7c64ad98ffe2a1dae9c3b3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:03:02.157965 14511 tablet_service.cc:1467] Processing CreateTablet for tablet dd8fdec0524549a39f607fd9d22b882a (DEFAULT_TABLE table=test-table [id=f9b719b7b7c64ad98ffe2a1dae9c3b3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:03:02.159122 14642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd8fdec0524549a39f607fd9d22b882a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:02.159232 14511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd8fdec0524549a39f607fd9d22b882a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:02.159232 14772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd8fdec0524549a39f607fd9d22b882a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:02.176723 14857 tablet_bootstrap.cc:492] T dd8fdec0524549a39f607fd9d22b882a P 3890a3226c704cbfa68f3b7698b5d93c: Bootstrap starting.
I20260926 15:03:02.178017 14855 tablet_bootstrap.cc:492] T dd8fdec0524549a39f607fd9d22b882a P 804c8ffefc5f4481b8f74840e8148c3a: Bootstrap starting.
I20260926 15:03:02.182184 14855 tablet_bootstrap.cc:654] T dd8fdec0524549a39f607fd9d22b882a P 804c8ffefc5f4481b8f74840e8148c3a: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:02.182403 14857 tablet_bootstrap.cc:654] T dd8fdec0524549a39f607fd9d22b882a P 3890a3226c704cbfa68f3b7698b5d93c: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:02.182431 14856 tablet_bootstrap.cc:492] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6: Bootstrap starting.
I20260926 15:03:02.183657 14855 log.cc:824] T dd8fdec0524549a39f607fd9d22b882a P 804c8ffefc5f4481b8f74840e8148c3a: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:02.183656 14857 log.cc:824] T dd8fdec0524549a39f607fd9d22b882a P 3890a3226c704cbfa68f3b7698b5d93c: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:02.186647 14856 tablet_bootstrap.cc:654] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:02.186992 14857 tablet_bootstrap.cc:492] T dd8fdec0524549a39f607fd9d22b882a P 3890a3226c704cbfa68f3b7698b5d93c: No bootstrap required, opened a new log
I20260926 15:03:02.187100 14855 tablet_bootstrap.cc:492] T dd8fdec0524549a39f607fd9d22b882a P 804c8ffefc5f4481b8f74840e8148c3a: No bootstrap required, opened a new log
I20260926 15:03:02.187160 14576 heartbeater.cc:503] Master 127.9.123.126:42229 was elected leader, sending a full tablet report...
I20260926 15:03:02.187322 14857 ts_tablet_manager.cc:1397] T dd8fdec0524549a39f607fd9d22b882a P 3890a3226c704cbfa68f3b7698b5d93c: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260926 15:03:02.187393 14855 ts_tablet_manager.cc:1397] T dd8fdec0524549a39f607fd9d22b882a P 804c8ffefc5f4481b8f74840e8148c3a: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260926 15:03:02.188030 14856 log.cc:824] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:02.196273 14856 tablet_bootstrap.cc:492] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6: No bootstrap required, opened a new log
I20260926 15:03:02.196627 14856 ts_tablet_manager.cc:1397] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20260926 15:03:02.196650 14857 raft_consensus.cc:348] T dd8fdec0524549a39f607fd9d22b882a P 3890a3226c704cbfa68f3b7698b5d93c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "804c8ffefc5f4481b8f74840e8148c3a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 36181 } } peers { permanent_uuid: "3890a3226c704cbfa68f3b7698b5d93c" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 34385 } } peers { permanent_uuid: "225f7bc90df54d4297d8d0adcb01d1c6" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34299 } }
I20260926 15:03:02.197080 14857 raft_consensus.cc:374] T dd8fdec0524549a39f607fd9d22b882a P 3890a3226c704cbfa68f3b7698b5d93c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:02.197237 14857 raft_consensus.cc:729] T dd8fdec0524549a39f607fd9d22b882a P 3890a3226c704cbfa68f3b7698b5d93c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3890a3226c704cbfa68f3b7698b5d93c, State: Initialized, Role: FOLLOWER
I20260926 15:03:02.197889 14857 consensus_queue.cc:260] T dd8fdec0524549a39f607fd9d22b882a P 3890a3226c704cbfa68f3b7698b5d93c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "804c8ffefc5f4481b8f74840e8148c3a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 36181 } } peers { permanent_uuid: "3890a3226c704cbfa68f3b7698b5d93c" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 34385 } } peers { permanent_uuid: "225f7bc90df54d4297d8d0adcb01d1c6" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34299 } }
I20260926 15:03:02.198475 14855 raft_consensus.cc:348] T dd8fdec0524549a39f607fd9d22b882a P 804c8ffefc5f4481b8f74840e8148c3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "804c8ffefc5f4481b8f74840e8148c3a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 36181 } } peers { permanent_uuid: "3890a3226c704cbfa68f3b7698b5d93c" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 34385 } } peers { permanent_uuid: "225f7bc90df54d4297d8d0adcb01d1c6" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34299 } }
I20260926 15:03:02.198906 14855 raft_consensus.cc:374] T dd8fdec0524549a39f607fd9d22b882a P 804c8ffefc5f4481b8f74840e8148c3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:02.199016 14855 raft_consensus.cc:729] T dd8fdec0524549a39f607fd9d22b882a P 804c8ffefc5f4481b8f74840e8148c3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 804c8ffefc5f4481b8f74840e8148c3a, State: Initialized, Role: FOLLOWER
I20260926 15:03:02.199643 14855 consensus_queue.cc:260] T dd8fdec0524549a39f607fd9d22b882a P 804c8ffefc5f4481b8f74840e8148c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "804c8ffefc5f4481b8f74840e8148c3a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 36181 } } peers { permanent_uuid: "3890a3226c704cbfa68f3b7698b5d93c" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 34385 } } peers { permanent_uuid: "225f7bc90df54d4297d8d0adcb01d1c6" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34299 } }
I20260926 15:03:02.206012 14707 heartbeater.cc:503] Master 127.9.123.126:42229 was elected leader, sending a full tablet report...
I20260926 15:03:02.206544 14837 heartbeater.cc:503] Master 127.9.123.126:42229 was elected leader, sending a full tablet report...
I20260926 15:03:02.207329 14857 ts_tablet_manager.cc:1428] T dd8fdec0524549a39f607fd9d22b882a P 3890a3226c704cbfa68f3b7698b5d93c: Time spent starting tablet: real 0.020s user 0.005s sys 0.014s
I20260926 15:03:02.208143 14855 ts_tablet_manager.cc:1428] T dd8fdec0524549a39f607fd9d22b882a P 804c8ffefc5f4481b8f74840e8148c3a: Time spent starting tablet: real 0.021s user 0.012s sys 0.006s
I20260926 15:03:02.208421 14856 raft_consensus.cc:348] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "804c8ffefc5f4481b8f74840e8148c3a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 36181 } } peers { permanent_uuid: "3890a3226c704cbfa68f3b7698b5d93c" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 34385 } } peers { permanent_uuid: "225f7bc90df54d4297d8d0adcb01d1c6" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34299 } }
I20260926 15:03:02.208808 14856 raft_consensus.cc:374] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:02.208959 14856 raft_consensus.cc:729] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 225f7bc90df54d4297d8d0adcb01d1c6, State: Initialized, Role: FOLLOWER
I20260926 15:03:02.209682 14856 consensus_queue.cc:260] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "804c8ffefc5f4481b8f74840e8148c3a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 36181 } } peers { permanent_uuid: "3890a3226c704cbfa68f3b7698b5d93c" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 34385 } } peers { permanent_uuid: "225f7bc90df54d4297d8d0adcb01d1c6" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34299 } }
I20260926 15:03:02.212024 14856 ts_tablet_manager.cc:1428] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
W20260926 15:03:02.308687 14838 tablet.cc:2369] T dd8fdec0524549a39f607fd9d22b882a P 3890a3226c704cbfa68f3b7698b5d93c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 15:03:02.375622 14708 tablet.cc:2369] T dd8fdec0524549a39f607fd9d22b882a P 804c8ffefc5f4481b8f74840e8148c3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 15:03:02.422732 14577 tablet.cc:2369] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:03:02.538653 14863 raft_consensus.cc:482] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:03:02.538987 14863 raft_consensus.cc:504] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "804c8ffefc5f4481b8f74840e8148c3a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 36181 } } peers { permanent_uuid: "3890a3226c704cbfa68f3b7698b5d93c" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 34385 } } peers { permanent_uuid: "225f7bc90df54d4297d8d0adcb01d1c6" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34299 } }
I20260926 15:03:02.540650 14863 leader_election.cc:288] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 804c8ffefc5f4481b8f74840e8148c3a (127.9.123.66:36181), 3890a3226c704cbfa68f3b7698b5d93c (127.9.123.67:34385)
I20260926 15:03:02.558212 14852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:02.541148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:40821 (local address 127.9.123.67:34385)
0926 15:03:02.541516 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:02.541521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:02.541875 (+ 354us) server_negotiation.cc:407] Connection header received
0926 15:03:02.542022 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:02.542028 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:02.542086 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:02.542176 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:02.557574 (+ 15398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:02.557579 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:02.557583 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:02.557603 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:02.557635 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:02.557647 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:02.557770 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:02.557866 (+ 96us) server_negotiation.cc:299] Negotiation successful
0926 15:03:02.557933 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262}
I20260926 15:03:02.558235 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:02.541127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:37897 (local address 127.9.123.66:36181)
0926 15:03:02.541383 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:02.541389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:02.542027 (+ 638us) server_negotiation.cc:407] Connection header received
0926 15:03:02.542150 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:02.542155 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:02.542214 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:02.542311 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:02.557565 (+ 15254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:02.557570 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:02.557573 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:02.557594 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:02.557627 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:02.557640 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:02.557751 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:02.557860 (+ 109us) server_negotiation.cc:299] Negotiation successful
0926 15:03:02.557921 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260926 15:03:02.558521 14865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:02.541118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:36181 (local address 127.9.123.65:37897)
0926 15:03:02.541914 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:02.541929 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:02.542038 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:02.542350 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:02.542353 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:02.542374 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:02.557299 (+ 14925us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 15:03:02.557302 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:02.557320 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 15:03:02.557371 (+ 51us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:02.557744 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:02.557749 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:02.557754 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 15:03:02.557951 (+ 197us) client_negotiation.cc:240] Negotiation successful
0926 15:03:02.558014 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":499,"threads_started":1}
I20260926 15:03:02.558668 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:02.540983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:34385 (local address 127.9.123.65:40821)
0926 15:03:02.541740 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:02.541768 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:02.541904 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:02.542215 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:02.542220 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:02.542242 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:02.557283 (+ 15041us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 15:03:02.557287 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:02.557306 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 15:03:02.557371 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:02.557741 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:02.557746 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:02.557751 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 15:03:02.558006 (+ 255us) client_negotiation.cc:240] Negotiation successful
0926 15:03:02.558065 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":375,"threads_started":1}
I20260926 15:03:02.559863 14792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8fdec0524549a39f607fd9d22b882a" candidate_uuid: "225f7bc90df54d4297d8d0adcb01d1c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3890a3226c704cbfa68f3b7698b5d93c" is_pre_election: true
I20260926 15:03:02.559867 14662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8fdec0524549a39f607fd9d22b882a" candidate_uuid: "225f7bc90df54d4297d8d0adcb01d1c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "804c8ffefc5f4481b8f74840e8148c3a" is_pre_election: true
I20260926 15:03:02.560452 14792 raft_consensus.cc:2454] T dd8fdec0524549a39f607fd9d22b882a P 3890a3226c704cbfa68f3b7698b5d93c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 225f7bc90df54d4297d8d0adcb01d1c6 in term 0.
I20260926 15:03:02.560452 14662 raft_consensus.cc:2454] T dd8fdec0524549a39f607fd9d22b882a P 804c8ffefc5f4481b8f74840e8148c3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 225f7bc90df54d4297d8d0adcb01d1c6 in term 0.
I20260926 15:03:02.561211 14467 leader_election.cc:302] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 225f7bc90df54d4297d8d0adcb01d1c6, 3890a3226c704cbfa68f3b7698b5d93c; no voters:
I20260926 15:03:02.561633 14863 raft_consensus.cc:2781] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:03:02.561784 14863 raft_consensus.cc:482] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:03:02.561885 14863 raft_consensus.cc:3037] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:02.564217 14863 raft_consensus.cc:504] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "804c8ffefc5f4481b8f74840e8148c3a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 36181 } } peers { permanent_uuid: "3890a3226c704cbfa68f3b7698b5d93c" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 34385 } } peers { permanent_uuid: "225f7bc90df54d4297d8d0adcb01d1c6" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34299 } }
I20260926 15:03:02.564966 14863 leader_election.cc:288] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6 [CANDIDATE]: Term 1 election: Requested vote from peers 804c8ffefc5f4481b8f74840e8148c3a (127.9.123.66:36181), 3890a3226c704cbfa68f3b7698b5d93c (127.9.123.67:34385)
I20260926 15:03:02.565508 14662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8fdec0524549a39f607fd9d22b882a" candidate_uuid: "225f7bc90df54d4297d8d0adcb01d1c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "804c8ffefc5f4481b8f74840e8148c3a"
I20260926 15:03:02.565616 14792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8fdec0524549a39f607fd9d22b882a" candidate_uuid: "225f7bc90df54d4297d8d0adcb01d1c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3890a3226c704cbfa68f3b7698b5d93c"
I20260926 15:03:02.565714 14662 raft_consensus.cc:3037] T dd8fdec0524549a39f607fd9d22b882a P 804c8ffefc5f4481b8f74840e8148c3a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:02.565843 14792 raft_consensus.cc:3037] T dd8fdec0524549a39f607fd9d22b882a P 3890a3226c704cbfa68f3b7698b5d93c [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:02.568214 14792 raft_consensus.cc:2454] T dd8fdec0524549a39f607fd9d22b882a P 3890a3226c704cbfa68f3b7698b5d93c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 225f7bc90df54d4297d8d0adcb01d1c6 in term 1.
I20260926 15:03:02.568311 14662 raft_consensus.cc:2454] T dd8fdec0524549a39f607fd9d22b882a P 804c8ffefc5f4481b8f74840e8148c3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 225f7bc90df54d4297d8d0adcb01d1c6 in term 1.
I20260926 15:03:02.568750 14467 leader_election.cc:302] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 225f7bc90df54d4297d8d0adcb01d1c6, 3890a3226c704cbfa68f3b7698b5d93c, 804c8ffefc5f4481b8f74840e8148c3a; no voters:
I20260926 15:03:02.569166 14863 raft_consensus.cc:2781] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:03:02.569689 14863 raft_consensus.cc:686] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6 [term 1 LEADER]: Becoming Leader. State: Replica: 225f7bc90df54d4297d8d0adcb01d1c6, State: Running, Role: LEADER
I20260926 15:03:02.570183 14863 consensus_queue.cc:237] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "804c8ffefc5f4481b8f74840e8148c3a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 36181 } } peers { permanent_uuid: "3890a3226c704cbfa68f3b7698b5d93c" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 34385 } } peers { permanent_uuid: "225f7bc90df54d4297d8d0adcb01d1c6" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34299 } }
I20260926 15:03:02.575346 14390 catalog_manager.cc:5495] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 225f7bc90df54d4297d8d0adcb01d1c6 (127.9.123.65). New cstate: current_term: 1 leader_uuid: "225f7bc90df54d4297d8d0adcb01d1c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "804c8ffefc5f4481b8f74840e8148c3a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 36181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3890a3226c704cbfa68f3b7698b5d93c" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 34385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "225f7bc90df54d4297d8d0adcb01d1c6" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 34299 } health_report { overall_health: HEALTHY } } }
I20260926 15:03:02.627391 14854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:02.624236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46172 (local address 127.9.123.65:34299)
0926 15:03:02.624464 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:02.624471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:02.625043 (+ 572us) server_negotiation.cc:407] Connection header received
0926 15:03:02.625292 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:02.625297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:02.625404 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:02.625504 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:02.626756 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:02.626763 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:02.626768 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:02.626789 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:02.626841 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:02.626857 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:02.626970 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:02.627059 (+ 89us) server_negotiation.cc:299] Negotiation successful
0926 15:03:02.627117 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260926 15:03:02.637507 14792 raft_consensus.cc:1261] T dd8fdec0524549a39f607fd9d22b882a P 3890a3226c704cbfa68f3b7698b5d93c [term 1 FOLLOWER]: Refusing update from remote peer 225f7bc90df54d4297d8d0adcb01d1c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:03:02.637507 14662 raft_consensus.cc:1261] T dd8fdec0524549a39f607fd9d22b882a P 804c8ffefc5f4481b8f74840e8148c3a [term 1 FOLLOWER]: Refusing update from remote peer 225f7bc90df54d4297d8d0adcb01d1c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:03:02.638470 14866 consensus_queue.cc:1035] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "804c8ffefc5f4481b8f74840e8148c3a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 36181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:03:02.638818 14863 consensus_queue.cc:1035] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3890a3226c704cbfa68f3b7698b5d93c" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 34385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:03:02.650705 14871 mvcc.cc:205] Tried to move back new op lower bound from 7333621688872349696 to 7333621688611381248. Current Snapshot: MvccSnapshot[applied={T|T < 7333621688872349696}]
I20260926 15:03:02.650830 14872 mvcc.cc:205] Tried to move back new op lower bound from 7333621688872349696 to 7333621688611381248. Current Snapshot: MvccSnapshot[applied={T|T < 7333621688872349696}]
I20260926 15:03:02.663292 14389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:03:02.663581 14389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:03:02.667292 14389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ea181dc3ec4847f6b265725049e891e1: Sending DeleteTablet for 3 replicas of tablet dd8fdec0524549a39f607fd9d22b882a
I20260926 15:03:02.668502 14772 tablet_service.cc:1514] Processing DeleteTablet for tablet dd8fdec0524549a39f607fd9d22b882a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:03:02 UTC) from {username='slave'} at 127.0.0.1:40310
I20260926 15:03:02.668504 14642 tablet_service.cc:1514] Processing DeleteTablet for tablet dd8fdec0524549a39f607fd9d22b882a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:03:02 UTC) from {username='slave'} at 127.0.0.1:52794
I20260926 15:03:02.669459 14880 tablet_replica.cc:330] T dd8fdec0524549a39f607fd9d22b882a P 804c8ffefc5f4481b8f74840e8148c3a: stopping tablet replica
I20260926 15:03:02.669530 14511 tablet_service.cc:1514] Processing DeleteTablet for tablet dd8fdec0524549a39f607fd9d22b882a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:03:02 UTC) from {username='slave'} at 127.0.0.1:46168
I20260926 15:03:02.669719 14881 tablet_replica.cc:330] T dd8fdec0524549a39f607fd9d22b882a P 3890a3226c704cbfa68f3b7698b5d93c: stopping tablet replica
I20260926 15:03:02.669948 14880 raft_consensus.cc:2229] T dd8fdec0524549a39f607fd9d22b882a P 804c8ffefc5f4481b8f74840e8148c3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:03:02.670127 14881 raft_consensus.cc:2229] T dd8fdec0524549a39f607fd9d22b882a P 3890a3226c704cbfa68f3b7698b5d93c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:03:02.670270 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 14449
I20260926 15:03:02.670529 14880 raft_consensus.cc:2258] T dd8fdec0524549a39f607fd9d22b882a P 804c8ffefc5f4481b8f74840e8148c3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:02.670792 14881 raft_consensus.cc:2258] T dd8fdec0524549a39f607fd9d22b882a P 3890a3226c704cbfa68f3b7698b5d93c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:02.672861 14880 ts_tablet_manager.cc:1905] T dd8fdec0524549a39f607fd9d22b882a P 804c8ffefc5f4481b8f74840e8148c3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:03:02.674485 14881 ts_tablet_manager.cc:1905] T dd8fdec0524549a39f607fd9d22b882a P 3890a3226c704cbfa68f3b7698b5d93c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:03:02.675491 14854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:02.672176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46176 (local address 127.9.123.65:34299)
0926 15:03:02.672372 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:02.672378 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:02.672846 (+ 468us) server_negotiation.cc:407] Connection header received
0926 15:03:02.673166 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:02.673172 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:02.673237 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:02.673331 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:02.674685 (+ 1354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:02.674691 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:02.674696 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:02.674715 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:02.674751 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:02.674766 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:02.674872 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:02.675136 (+ 264us) server_negotiation.cc:299] Negotiation successful
0926 15:03:02.675199 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 15:03:02.676702 14571 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:02.680532 14880 ts_tablet_manager.cc:1918] T dd8fdec0524549a39f607fd9d22b882a P 804c8ffefc5f4481b8f74840e8148c3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:03:02.680809 14880 log.cc:1196] T dd8fdec0524549a39f607fd9d22b882a P 804c8ffefc5f4481b8f74840e8148c3a: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/dd8fdec0524549a39f607fd9d22b882a
I20260926 15:03:02.681422 14880 ts_tablet_manager.cc:1939] T dd8fdec0524549a39f607fd9d22b882a P 804c8ffefc5f4481b8f74840e8148c3a: Deleting consensus metadata
I20260926 15:03:02.683010 14881 ts_tablet_manager.cc:1918] T dd8fdec0524549a39f607fd9d22b882a P 3890a3226c704cbfa68f3b7698b5d93c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:03:02.683326 14881 log.cc:1196] T dd8fdec0524549a39f607fd9d22b882a P 3890a3226c704cbfa68f3b7698b5d93c: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/dd8fdec0524549a39f607fd9d22b882a
I20260926 15:03:02.683650 14374 catalog_manager.cc:4841] TS 804c8ffefc5f4481b8f74840e8148c3a (127.9.123.66:36181): tablet dd8fdec0524549a39f607fd9d22b882a (table test-table [id=f9b719b7b7c64ad98ffe2a1dae9c3b3a]) successfully deleted
I20260926 15:03:02.683950 14881 ts_tablet_manager.cc:1939] T dd8fdec0524549a39f607fd9d22b882a P 3890a3226c704cbfa68f3b7698b5d93c: Deleting consensus metadata
I20260926 15:03:02.686019 14377 catalog_manager.cc:4841] TS 3890a3226c704cbfa68f3b7698b5d93c (127.9.123.67:34385): tablet dd8fdec0524549a39f607fd9d22b882a (table test-table [id=f9b719b7b7c64ad98ffe2a1dae9c3b3a]) successfully deleted
I20260926 15:03:02.781898 14882 tablet_replica.cc:330] T dd8fdec0524549a39f607fd9d22b882a P 225f7bc90df54d4297d8d0adcb01d1c6: stopping tablet replica
I20260926 15:03:02.782228 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 14449
W20260926 15:03:02.796312 14376 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.123.65:34299 (error 108)
I20260926 15:03:02.796852 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 14580
I20260926 15:03:02.798341 14886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:02.797597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:34299 (local address 127.0.0.1:46182)
0926 15:03:02.797993 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:02.798127 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:34299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":196,"threads_started":1}
W20260926 15:03:02.798561 14376 catalog_manager.cc:4584] TS 225f7bc90df54d4297d8d0adcb01d1c6 (127.9.123.65:34299): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dd8fdec0524549a39f607fd9d22b882a: Network error: Client connection negotiation failed: client connection to 127.9.123.65:34299: connect: Connection refused (error 111)
I20260926 15:03:02.799657 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:02.797603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52802 (local address 127.9.123.66:36181)
0926 15:03:02.797793 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:02.797799 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:02.797820 (+ 21us) server_negotiation.cc:407] Connection header received
0926 15:03:02.798018 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:02.798023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:02.798117 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:02.798223 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:02.799153 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:02.799158 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:02.799161 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:02.799175 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:02.799205 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:02.799216 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:02.799309 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:02.799359 (+ 50us) server_negotiation.cc:299] Negotiation successful
0926 15:03:02.799405 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 15:03:02.800246 14702 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:02.852090 14886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:02.851613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:34299 (local address 127.0.0.1:46192)
0926 15:03:02.851848 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:02.851956 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:34299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260926 15:03:02.853075 14886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:02.852720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:34299 (local address 127.0.0.1:46206)
0926 15:03:02.852865 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:02.852949 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:34299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260926 15:03:02.886143 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 14580
I20260926 15:03:02.899624 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 14710
I20260926 15:03:02.902202 14852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:02.900361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40316 (local address 127.9.123.67:34385)
0926 15:03:02.900520 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:02.900524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:02.900543 (+ 19us) server_negotiation.cc:407] Connection header received
0926 15:03:02.900605 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:02.900608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:02.900695 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:02.900790 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:02.901701 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:02.901705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:02.901708 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:02.901723 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:02.901755 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:02.901766 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:02.901900 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:02.901933 (+ 33us) server_negotiation.cc:299] Negotiation successful
0926 15:03:02.901978 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 15:03:02.902820 14832 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:02.913754 14886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:02.913217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:34299 (local address 127.0.0.1:46210)
0926 15:03:02.913450 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:02.913567 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:34299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260926 15:03:02.914851 14886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:02.914411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:34299 (local address 127.0.0.1:46220)
0926 15:03:02.914577 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:02.914676 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:34299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260926 15:03:02.987864 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 14710
I20260926 15:03:02.995157 14886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:02.994716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:34299 (local address 127.0.0.1:46230)
0926 15:03:02.994894 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:02.994980 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:34299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260926 15:03:02.996263 14886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:02.995843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:34299 (local address 127.0.0.1:46238)
0926 15:03:02.996058 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:02.996127 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:34299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260926 15:03:03.000764 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 14359
I20260926 15:03:03.001963 14420 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:03.104247 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 14359
2026-09-26T15:03:03Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2958 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260926 15:03:03.143149 9709 mini_oidc.cc:152] Starting JWKS server
I20260926 15:03:03.145874 9709 webserver.cc:460] Webserver started at https://127.0.0.1:33243/ using document root <none> and password file <none>
I20260926 15:03:03.146179 9709 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 15:03:03.146848 9709 webserver.cc:460] Webserver started at http://127.0.0.1:34359/ using document root <none> and password file <none>
2026-09-26T15:03:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:03:03Z Disabled control of system clock
I20260926 15:03:03.172402 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:39677
--webserver_interface=127.9.123.126
--webserver_port=0
--builtin_ntp_servers=127.9.123.84:42833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:39677
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33243/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434939095847-9709-0/testchainca.cert with env {}
W20260926 15:03:03.497979 14895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:03.498297 14895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:03.498430 14895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:03.505744 14895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:03:03.505887 14895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:03.505951 14895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:03.506031 14895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:03:03.506090 14895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 15:03:03.506173 14895 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 15:03:03.506225 14895 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33243/jwks.json
I20260926 15:03:03.516312 14895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:42833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:39677
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:39677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33243/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434939095847-9709-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:03.517721 14895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:03.519120 14895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:03.532583 14900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:03.532583 14901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:03.532676 14903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:03.533418 14895 server_base.cc:1034] running on GCE node
I20260926 15:03:03.534231 14895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:03.535727 14895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:03.536932 14895 hybrid_clock.cc:648] HybridClock initialized: now 1790434983536887 us; error 45 us; skew 500 ppm
I20260926 15:03:03.537325 14895 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:03:03.539559 14895 webserver.cc:460] Webserver started at http://127.9.123.126:39037/ using document root <none> and password file <none>
I20260926 15:03:03.540257 14895 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:03.540375 14895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:03.540666 14895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:03.544144 14895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "635c19707ffe4528a54bae96dda2ced1"
format_stamp: "Formatted at 2026-09-26 15:03:03 on dist-test-slave-7153"
server_key: "504776a9a83c47623073ee39d7a66c4d"
server_key_iv: "bcecee5777a029ac621ba0f647f08675"
server_key_version: "encryptionkey@0"
I20260926 15:03:03.545148 14895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "635c19707ffe4528a54bae96dda2ced1"
format_stamp: "Formatted at 2026-09-26 15:03:03 on dist-test-slave-7153"
server_key: "504776a9a83c47623073ee39d7a66c4d"
server_key_iv: "bcecee5777a029ac621ba0f647f08675"
server_key_version: "encryptionkey@0"
I20260926 15:03:03.551335 14895 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260926 15:03:03.555711 14909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:03.557472 14895 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:03:03.557682 14895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "635c19707ffe4528a54bae96dda2ced1"
format_stamp: "Formatted at 2026-09-26 15:03:03 on dist-test-slave-7153"
server_key: "504776a9a83c47623073ee39d7a66c4d"
server_key_iv: "bcecee5777a029ac621ba0f647f08675"
server_key_version: "encryptionkey@0"
I20260926 15:03:03.557937 14895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:03.578367 14895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:03.579236 14895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:03.579523 14895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:03.596294 14895 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:39677
I20260926 15:03:03.596310 14960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:39677 every 8 connection(s)
I20260926 15:03:03.597647 14895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:03:03.601136 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 14895
I20260926 15:03:03.601356 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:03:03.601648 14961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:03.602139 9709 external_mini_cluster.cc:1428] Setting key 7a6d5c8382166d481a59c413fd8c4667
I20260926 15:03:03.614993 14961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 635c19707ffe4528a54bae96dda2ced1: Bootstrap starting.
I20260926 15:03:03.615818 14964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:03.604143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53604 (local address 127.9.123.126:39677)
0926 15:03:03.605187 (+ 1044us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:03.605198 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:03.605244 (+ 46us) server_negotiation.cc:407] Connection header received
0926 15:03:03.606859 (+ 1615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:03.606889 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:03.607420 (+ 531us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:03.607812 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:03.608452 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:03.609831 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:03.611289 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:03.612013 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:03.612196 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:03.613443 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:03.613451 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:03.613467 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:03.614019 (+ 552us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:03.614099 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:03.614121 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:03.614218 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:03.614446 (+ 228us) server_negotiation.cc:299] Negotiation successful
0926 15:03:03.614768 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":323,"threads_started":1}
I20260926 15:03:03.618307 14961 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 635c19707ffe4528a54bae96dda2ced1: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:03.619412 14961 log.cc:824] T 00000000000000000000000000000000 P 635c19707ffe4528a54bae96dda2ced1: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:03.619683 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:03:03.622326 14961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 635c19707ffe4528a54bae96dda2ced1: No bootstrap required, opened a new log
I20260926 15:03:03.629876 14961 raft_consensus.cc:348] T 00000000000000000000000000000000 P 635c19707ffe4528a54bae96dda2ced1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635c19707ffe4528a54bae96dda2ced1" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39677 } }
I20260926 15:03:03.630270 14961 raft_consensus.cc:374] T 00000000000000000000000000000000 P 635c19707ffe4528a54bae96dda2ced1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:03.630374 14961 raft_consensus.cc:729] T 00000000000000000000000000000000 P 635c19707ffe4528a54bae96dda2ced1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 635c19707ffe4528a54bae96dda2ced1, State: Initialized, Role: FOLLOWER
I20260926 15:03:03.630956 14961 consensus_queue.cc:260] T 00000000000000000000000000000000 P 635c19707ffe4528a54bae96dda2ced1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635c19707ffe4528a54bae96dda2ced1" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39677 } }
I20260926 15:03:03.631163 14961 raft_consensus.cc:388] T 00000000000000000000000000000000 P 635c19707ffe4528a54bae96dda2ced1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:03:03.631287 14961 raft_consensus.cc:482] T 00000000000000000000000000000000 P 635c19707ffe4528a54bae96dda2ced1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:03:03.631456 14961 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 635c19707ffe4528a54bae96dda2ced1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:03.633641 14961 raft_consensus.cc:504] T 00000000000000000000000000000000 P 635c19707ffe4528a54bae96dda2ced1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635c19707ffe4528a54bae96dda2ced1" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39677 } }
I20260926 15:03:03.634159 14961 leader_election.cc:302] T 00000000000000000000000000000000 P 635c19707ffe4528a54bae96dda2ced1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 635c19707ffe4528a54bae96dda2ced1; no voters:
I20260926 15:03:03.634742 14961 leader_election.cc:288] T 00000000000000000000000000000000 P 635c19707ffe4528a54bae96dda2ced1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:03:03.634893 14966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 635c19707ffe4528a54bae96dda2ced1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:03:03.635529 14966 raft_consensus.cc:686] T 00000000000000000000000000000000 P 635c19707ffe4528a54bae96dda2ced1 [term 1 LEADER]: Becoming Leader. State: Replica: 635c19707ffe4528a54bae96dda2ced1, State: Running, Role: LEADER
I20260926 15:03:03.636109 14966 consensus_queue.cc:237] T 00000000000000000000000000000000 P 635c19707ffe4528a54bae96dda2ced1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635c19707ffe4528a54bae96dda2ced1" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39677 } }
I20260926 15:03:03.636724 14961 sys_catalog.cc:564] T 00000000000000000000000000000000 P 635c19707ffe4528a54bae96dda2ced1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:03:03.638844 14968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 635c19707ffe4528a54bae96dda2ced1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 635c19707ffe4528a54bae96dda2ced1. Latest consensus state: current_term: 1 leader_uuid: "635c19707ffe4528a54bae96dda2ced1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635c19707ffe4528a54bae96dda2ced1" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39677 } } }
I20260926 15:03:03.639237 14968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 635c19707ffe4528a54bae96dda2ced1 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:03.639324 14967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 635c19707ffe4528a54bae96dda2ced1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "635c19707ffe4528a54bae96dda2ced1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "635c19707ffe4528a54bae96dda2ced1" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39677 } } }
I20260926 15:03:03.639653 14967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 635c19707ffe4528a54bae96dda2ced1 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:03.640969 14975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:03:03.645319 14975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:03:03.653944 14975 catalog_manager.cc:1347] Generated new cluster ID: 817e8d4553c14e1da2ba9af1f0c49423
I20260926 15:03:03.654072 14975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:03:03.671538 14975 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:03:03.672940 14975 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:03:03.680994 14975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 635c19707ffe4528a54bae96dda2ced1: Generated new TSK 0
I20260926 15:03:03.681917 14975 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 15:03:03.720604 14991 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260926 15:03:03.720796 14991 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 15:03:03.721920 14964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:03.701717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53616 (local address 127.9.123.126:39677)
0926 15:03:03.701874 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:03.701879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:03.702417 (+ 538us) server_negotiation.cc:407] Connection header received
0926 15:03:03.702730 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:03.702734 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:03.702813 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:03.702930 (+ 117us) server_negotiation.cc:226] Negotiated authn=JWT
0926 15:03:03.703472 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:03.704200 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:03.705641 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:03.706092 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:03.706203 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:03.706730 (+ 527us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 15:03:03.721230 (+ 14500us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33243/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 15:03:03.721587 (+ 357us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53616: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33243/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":56}
W20260926 15:03:03.722095 14964 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53616: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33243/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260926 15:03:03.723415 14990 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:03.701696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39677 (local address 127.0.0.1:53616)
0926 15:03:03.702309 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:03.702325 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:03.702507 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:03.702979 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:03.702984 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:03.702997 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0926 15:03:03.703312 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:03.703320 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:03.704336 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:03.704340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:03.705505 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:03.705517 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:03.706270 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:03.706277 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:03.706398 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:03.706471 (+ 73us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 15:03:03.721532 (+ 15061us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33243/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0926 15:03:03.721892 (+ 360us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.123.126:39677: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33243/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":318,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.9.123.126:39677: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33243/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260926 15:03:03.725683 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 14895
I20260926 15:03:03.726877 14956 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:03.818842 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 14895
2026-09-26T15:03:03Z chronyd exiting
I20260926 15:03:03.865833 9709 test_util.cc:175] -----------------------------------------------
I20260926 15:03:03.865980 9709 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790434939095847-9709-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (727 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260926 15:03:03.869719 9709 mini_oidc.cc:152] Starting JWKS server
I20260926 15:03:03.871505 9709 webserver.cc:460] Webserver started at https://127.0.0.1:36011/ using document root <none> and password file <none>
I20260926 15:03:03.871719 9709 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 15:03:03.872399 9709 webserver.cc:460] Webserver started at http://127.0.0.1:39821/ using document root <none> and password file <none>
2026-09-26T15:03:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:03:03Z Disabled control of system clock
I20260926 15:03:03.898224 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:36285
--webserver_interface=127.9.123.126
--webserver_port=0
--builtin_ntp_servers=127.9.123.84:41901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:36285
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36011/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434939095847-9709-0/testchainca.cert with env {}
W20260926 15:03:04.219213 14998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:04.219534 14998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:04.219676 14998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:04.227248 14998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:03:04.227387 14998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:04.227460 14998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:04.227535 14998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:03:04.227591 14998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 15:03:04.227672 14998 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 15:03:04.227721 14998 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36011/jwks.json
I20260926 15:03:04.238144 14998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:41901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:36285
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:36285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36011/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434939095847-9709-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:04.239540 14998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:04.240859 14998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:04.252839 15006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:04.252884 15003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:04.253553 15004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:04.254444 14998 server_base.cc:1034] running on GCE node
I20260926 15:03:04.255486 14998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:04.257007 14998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:04.258234 14998 hybrid_clock.cc:648] HybridClock initialized: now 1790434984258180 us; error 45 us; skew 500 ppm
I20260926 15:03:04.258608 14998 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:03:04.260860 14998 webserver.cc:460] Webserver started at http://127.9.123.126:39785/ using document root <none> and password file <none>
I20260926 15:03:04.261590 14998 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:04.261696 14998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:04.262004 14998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:04.264886 14998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "9babdac733734e70a8d50a6be02ab2f3"
format_stamp: "Formatted at 2026-09-26 15:03:04 on dist-test-slave-7153"
server_key: "0bbf25a9821b5177d8cd08cb34687203"
server_key_iv: "b0531d32b6cd5809a6502b7a4a6b22ba"
server_key_version: "encryptionkey@0"
I20260926 15:03:04.265733 14998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "9babdac733734e70a8d50a6be02ab2f3"
format_stamp: "Formatted at 2026-09-26 15:03:04 on dist-test-slave-7153"
server_key: "0bbf25a9821b5177d8cd08cb34687203"
server_key_iv: "b0531d32b6cd5809a6502b7a4a6b22ba"
server_key_version: "encryptionkey@0"
I20260926 15:03:04.271009 14998 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 15:03:04.274678 15012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:04.276212 14998 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 15:03:04.276386 14998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "9babdac733734e70a8d50a6be02ab2f3"
format_stamp: "Formatted at 2026-09-26 15:03:04 on dist-test-slave-7153"
server_key: "0bbf25a9821b5177d8cd08cb34687203"
server_key_iv: "b0531d32b6cd5809a6502b7a4a6b22ba"
server_key_version: "encryptionkey@0"
I20260926 15:03:04.276618 14998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:04.288480 14998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:04.289384 14998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:04.289683 14998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:04.307094 14998 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:36285
I20260926 15:03:04.307111 15063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:36285 every 8 connection(s)
I20260926 15:03:04.308519 14998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:03:04.312422 15064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:04.317013 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 14998
I20260926 15:03:04.317235 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:03:04.317873 9709 external_mini_cluster.cc:1428] Setting key 21950f83a8317b5df2e722e11e425829
I20260926 15:03:04.324657 15064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9babdac733734e70a8d50a6be02ab2f3: Bootstrap starting.
I20260926 15:03:04.328620 15064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9babdac733734e70a8d50a6be02ab2f3: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:04.330065 15064 log.cc:824] T 00000000000000000000000000000000 P 9babdac733734e70a8d50a6be02ab2f3: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:04.333372 15064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9babdac733734e70a8d50a6be02ab2f3: No bootstrap required, opened a new log
I20260926 15:03:04.333681 15067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:04.319867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47200 (local address 127.9.123.126:36285)
0926 15:03:04.321061 (+ 1194us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:04.321083 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:04.321169 (+ 86us) server_negotiation.cc:407] Connection header received
0926 15:03:04.323494 (+ 2325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:04.323527 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:04.324222 (+ 695us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:04.324737 (+ 515us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:04.325380 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:04.327391 (+ 2011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:04.328831 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:04.329938 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:04.330132 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:04.331196 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:04.331206 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:04.331231 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:04.331882 (+ 651us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:04.331958 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:04.331981 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:04.332075 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:04.332291 (+ 216us) server_negotiation.cc:299] Negotiation successful
0926 15:03:04.332642 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":341,"threads_started":1}
I20260926 15:03:04.337284 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:03:04.341032 15064 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9babdac733734e70a8d50a6be02ab2f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9babdac733734e70a8d50a6be02ab2f3" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 36285 } }
I20260926 15:03:04.341426 15064 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9babdac733734e70a8d50a6be02ab2f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:04.341524 15064 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9babdac733734e70a8d50a6be02ab2f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9babdac733734e70a8d50a6be02ab2f3, State: Initialized, Role: FOLLOWER
I20260926 15:03:04.342100 15064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9babdac733734e70a8d50a6be02ab2f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9babdac733734e70a8d50a6be02ab2f3" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 36285 } }
I20260926 15:03:04.342308 15064 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9babdac733734e70a8d50a6be02ab2f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:03:04.342432 15064 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9babdac733734e70a8d50a6be02ab2f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:03:04.342581 15064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9babdac733734e70a8d50a6be02ab2f3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:04.344954 15064 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9babdac733734e70a8d50a6be02ab2f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9babdac733734e70a8d50a6be02ab2f3" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 36285 } }
I20260926 15:03:04.345484 15064 leader_election.cc:302] T 00000000000000000000000000000000 P 9babdac733734e70a8d50a6be02ab2f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9babdac733734e70a8d50a6be02ab2f3; no voters:
I20260926 15:03:04.346020 15064 leader_election.cc:288] T 00000000000000000000000000000000 P 9babdac733734e70a8d50a6be02ab2f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:03:04.346127 15069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9babdac733734e70a8d50a6be02ab2f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:03:04.346750 15069 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9babdac733734e70a8d50a6be02ab2f3 [term 1 LEADER]: Becoming Leader. State: Replica: 9babdac733734e70a8d50a6be02ab2f3, State: Running, Role: LEADER
I20260926 15:03:04.347270 15069 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9babdac733734e70a8d50a6be02ab2f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9babdac733734e70a8d50a6be02ab2f3" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 36285 } }
I20260926 15:03:04.348001 15064 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9babdac733734e70a8d50a6be02ab2f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:03:04.350035 15070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9babdac733734e70a8d50a6be02ab2f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9babdac733734e70a8d50a6be02ab2f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9babdac733734e70a8d50a6be02ab2f3" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 36285 } } }
I20260926 15:03:04.350080 15071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9babdac733734e70a8d50a6be02ab2f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9babdac733734e70a8d50a6be02ab2f3. Latest consensus state: current_term: 1 leader_uuid: "9babdac733734e70a8d50a6be02ab2f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9babdac733734e70a8d50a6be02ab2f3" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 36285 } } }
I20260926 15:03:04.350471 15071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9babdac733734e70a8d50a6be02ab2f3 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:04.350479 15070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9babdac733734e70a8d50a6be02ab2f3 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:04.351555 15076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:03:04.360921 15076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:03:04.370113 15076 catalog_manager.cc:1347] Generated new cluster ID: de48dc586715489b8e30ed3dbc031bc1
I20260926 15:03:04.370234 15076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:03:04.396276 15076 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:03:04.398274 15076 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:03:04.412477 15076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9babdac733734e70a8d50a6be02ab2f3: Generated new TSK 0
I20260926 15:03:04.413645 15076 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260926 15:03:04.455947 15094 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260926 15:03:04.456127 15094 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 15:03:04.457031 15067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:04.429697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47206 (local address 127.9.123.126:36285)
0926 15:03:04.429915 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:04.429922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:04.430300 (+ 378us) server_negotiation.cc:407] Connection header received
0926 15:03:04.430558 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:04.430563 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:04.430633 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:04.430768 (+ 135us) server_negotiation.cc:226] Negotiated authn=JWT
0926 15:03:04.431242 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:04.431969 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:04.434004 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:04.434494 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:04.434587 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:04.435130 (+ 543us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 15:03:04.456454 (+ 21324us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36011/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 15:03:04.456796 (+ 342us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47206: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36011/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":54}
W20260926 15:03:04.457176 15067 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47206: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36011/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260926 15:03:04.457466 15093 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:04.429506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:36285 (local address 127.0.0.1:47206)
0926 15:03:04.430181 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:04.430205 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:04.430361 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:04.430799 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:04.430804 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:04.430817 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0926 15:03:04.431106 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:04.431118 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:04.432106 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:04.432112 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:04.433856 (+ 1744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:04.433871 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:04.434649 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:04.434656 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:04.434850 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:04.434896 (+ 46us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 15:03:04.456708 (+ 21812us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36011/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0926 15:03:04.457002 (+ 294us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.123.126:36285: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36011/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":326,"threads_started":1}
I20260926 15:03:04.458877 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 14998
I20260926 15:03:04.460048 15059 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:04.552305 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 14998
2026-09-26T15:03:04Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (733 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260926 15:03:04.603724 9709 mini_oidc.cc:152] Starting JWKS server
I20260926 15:03:04.605674 9709 webserver.cc:460] Webserver started at https://127.0.0.1:41407/ using document root <none> and password file <none>
I20260926 15:03:04.605953 9709 mini_oidc.cc:189] Starting OIDC Discovery server
I20260926 15:03:04.606617 9709 webserver.cc:460] Webserver started at http://127.0.0.1:41185/ using document root <none> and password file <none>
2026-09-26T15:03:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:03:04Z Disabled control of system clock
I20260926 15:03:04.632437 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:46103
--webserver_interface=127.9.123.126
--webserver_port=0
--builtin_ntp_servers=127.9.123.84:39999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:46103
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41407/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 15:03:04.947337 15101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:04.947638 15101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:04.947721 15101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:04.955173 15101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:03:04.955356 15101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:04.955399 15101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:04.955457 15101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:03:04.955508 15101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 15:03:04.955564 15101 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 15:03:04.955605 15101 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41407/jwks.json
I20260926 15:03:04.965682 15101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:39999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:46103
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:46103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41407/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:04.967033 15101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:04.968850 15101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:04.981606 15107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:04.981609 15109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:04.982236 15101 server_base.cc:1034] running on GCE node
W20260926 15:03:04.982333 15106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:04.983292 15101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:04.984822 15101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:04.986073 15101 hybrid_clock.cc:648] HybridClock initialized: now 1790434984985977 us; error 49 us; skew 500 ppm
I20260926 15:03:04.986416 15101 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:03:04.988786 15101 webserver.cc:460] Webserver started at http://127.9.123.126:38693/ using document root <none> and password file <none>
I20260926 15:03:04.989550 15101 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:04.989641 15101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:04.989919 15101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:04.992866 15101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "050d8d164d8c47c893cea6217cdcfcc5"
format_stamp: "Formatted at 2026-09-26 15:03:04 on dist-test-slave-7153"
server_key: "e1c0192c9bbe074fc394494995605062"
server_key_iv: "b8a0f551f2decc882258056adf064096"
server_key_version: "encryptionkey@0"
I20260926 15:03:04.993717 15101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "050d8d164d8c47c893cea6217cdcfcc5"
format_stamp: "Formatted at 2026-09-26 15:03:04 on dist-test-slave-7153"
server_key: "e1c0192c9bbe074fc394494995605062"
server_key_iv: "b8a0f551f2decc882258056adf064096"
server_key_version: "encryptionkey@0"
I20260926 15:03:04.998955 15101 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 15:03:05.002555 15115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:05.003952 15101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260926 15:03:05.004132 15101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "050d8d164d8c47c893cea6217cdcfcc5"
format_stamp: "Formatted at 2026-09-26 15:03:04 on dist-test-slave-7153"
server_key: "e1c0192c9bbe074fc394494995605062"
server_key_iv: "b8a0f551f2decc882258056adf064096"
server_key_version: "encryptionkey@0"
I20260926 15:03:05.004317 15101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:05.042502 15101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:05.043377 15101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:05.043666 15101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:05.060981 15101 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:46103
I20260926 15:03:05.061005 15166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:46103 every 8 connection(s)
I20260926 15:03:05.062405 15101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:03:05.066162 15167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:05.071964 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 15101
I20260926 15:03:05.072186 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:03:05.072767 9709 external_mini_cluster.cc:1428] Setting key cbea3306b1942d65e9be6363bf4a7a48
I20260926 15:03:05.077466 15167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 050d8d164d8c47c893cea6217cdcfcc5: Bootstrap starting.
I20260926 15:03:05.081240 15167 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 050d8d164d8c47c893cea6217cdcfcc5: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:05.082607 15167 log.cc:824] T 00000000000000000000000000000000 P 050d8d164d8c47c893cea6217cdcfcc5: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:05.086158 15167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 050d8d164d8c47c893cea6217cdcfcc5: No bootstrap required, opened a new log
I20260926 15:03:05.087451 15170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:05.074870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39414 (local address 127.9.123.126:46103)
0926 15:03:05.075985 (+ 1115us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:05.075998 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:05.076046 (+ 48us) server_negotiation.cc:407] Connection header received
0926 15:03:05.077817 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:05.077844 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:05.078377 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:05.078780 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:05.079386 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:05.080649 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:05.082531 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:05.083436 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:05.083661 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:05.084912 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:05.084920 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:05.084936 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:05.085583 (+ 647us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:05.085669 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:05.085692 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:05.085792 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:05.086065 (+ 273us) server_negotiation.cc:299] Negotiation successful
0926 15:03:05.086408 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":380,"threads_started":1}
I20260926 15:03:05.091550 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:03:05.094198 15167 raft_consensus.cc:348] T 00000000000000000000000000000000 P 050d8d164d8c47c893cea6217cdcfcc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050d8d164d8c47c893cea6217cdcfcc5" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 46103 } }
I20260926 15:03:05.094599 15167 raft_consensus.cc:374] T 00000000000000000000000000000000 P 050d8d164d8c47c893cea6217cdcfcc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:05.094702 15167 raft_consensus.cc:729] T 00000000000000000000000000000000 P 050d8d164d8c47c893cea6217cdcfcc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 050d8d164d8c47c893cea6217cdcfcc5, State: Initialized, Role: FOLLOWER
I20260926 15:03:05.095314 15167 consensus_queue.cc:260] T 00000000000000000000000000000000 P 050d8d164d8c47c893cea6217cdcfcc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050d8d164d8c47c893cea6217cdcfcc5" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 46103 } }
I20260926 15:03:05.095534 15167 raft_consensus.cc:388] T 00000000000000000000000000000000 P 050d8d164d8c47c893cea6217cdcfcc5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:03:05.095630 15167 raft_consensus.cc:482] T 00000000000000000000000000000000 P 050d8d164d8c47c893cea6217cdcfcc5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:03:05.095750 15167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 050d8d164d8c47c893cea6217cdcfcc5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:05.097950 15167 raft_consensus.cc:504] T 00000000000000000000000000000000 P 050d8d164d8c47c893cea6217cdcfcc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050d8d164d8c47c893cea6217cdcfcc5" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 46103 } }
I20260926 15:03:05.098524 15167 leader_election.cc:302] T 00000000000000000000000000000000 P 050d8d164d8c47c893cea6217cdcfcc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 050d8d164d8c47c893cea6217cdcfcc5; no voters:
I20260926 15:03:05.099155 15167 leader_election.cc:288] T 00000000000000000000000000000000 P 050d8d164d8c47c893cea6217cdcfcc5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:03:05.099315 15172 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 050d8d164d8c47c893cea6217cdcfcc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:03:05.100023 15172 raft_consensus.cc:686] T 00000000000000000000000000000000 P 050d8d164d8c47c893cea6217cdcfcc5 [term 1 LEADER]: Becoming Leader. State: Replica: 050d8d164d8c47c893cea6217cdcfcc5, State: Running, Role: LEADER
I20260926 15:03:05.100579 15172 consensus_queue.cc:237] T 00000000000000000000000000000000 P 050d8d164d8c47c893cea6217cdcfcc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050d8d164d8c47c893cea6217cdcfcc5" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 46103 } }
I20260926 15:03:05.101338 15167 sys_catalog.cc:564] T 00000000000000000000000000000000 P 050d8d164d8c47c893cea6217cdcfcc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:03:05.103274 15173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 050d8d164d8c47c893cea6217cdcfcc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "050d8d164d8c47c893cea6217cdcfcc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050d8d164d8c47c893cea6217cdcfcc5" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 46103 } } }
I20260926 15:03:05.103703 15173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 050d8d164d8c47c893cea6217cdcfcc5 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:05.104859 15174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 050d8d164d8c47c893cea6217cdcfcc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 050d8d164d8c47c893cea6217cdcfcc5. Latest consensus state: current_term: 1 leader_uuid: "050d8d164d8c47c893cea6217cdcfcc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "050d8d164d8c47c893cea6217cdcfcc5" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 46103 } } }
I20260926 15:03:05.105314 15174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 050d8d164d8c47c893cea6217cdcfcc5 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:05.106797 15178 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:03:05.112360 15178 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:03:05.121769 15178 catalog_manager.cc:1347] Generated new cluster ID: 8df7e025ad2747a4b222da177ec626c9
I20260926 15:03:05.121912 15178 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:03:05.139642 15178 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:03:05.141604 15178 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:03:05.161221 15178 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 050d8d164d8c47c893cea6217cdcfcc5: Generated new TSK 0
I20260926 15:03:05.162102 15178 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:03:05.188915 15170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:05.180264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39424 (local address 127.9.123.126:46103)
0926 15:03:05.180462 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:05.180468 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:05.180873 (+ 405us) server_negotiation.cc:407] Connection header received
0926 15:03:05.181127 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:05.181133 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:05.181234 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:05.181403 (+ 169us) server_negotiation.cc:226] Negotiated authn=JWT
0926 15:03:05.181898 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:05.183080 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:05.184501 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:05.185223 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:05.185352 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:05.185852 (+ 500us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 15:03:05.188262 (+ 2410us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41407/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 15:03:05.188589 (+ 327us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39424: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41407/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":62}
W20260926 15:03:05.189065 15170 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39424: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41407/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260926 15:03:05.190552 15196 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:05.180152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:46103 (local address 127.0.0.1:39424)
0926 15:03:05.180770 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:05.180786 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:05.180918 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:05.181456 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:05.181460 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:05.181472 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0926 15:03:05.181748 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:05.181756 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:05.183263 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:05.183267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:05.184350 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:05.184363 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:05.185401 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:05.185406 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:05.185539 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:05.185576 (+ 37us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 15:03:05.188475 (+ 2899us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41407/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0926 15:03:05.190135 (+ 1660us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.123.126:46103: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41407/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":483,"mutex_wait_us":1358,"thread_start_us":338,"threads_started":1}
I20260926 15:03:05.192642 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 15101
I20260926 15:03:05.193912 15162 generic_service.cc:196] Checking for leaks (request via RPC)
E20260926 15:03:05.200459 15197 webserver.cc:533] Webserver: sslize failed:
E20260926 15:03:05.200577 15197 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260926 15:03:05.283713 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 15101
2026-09-26T15:03:05Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (731 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-26T15:03:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:03:05Z Disabled control of system clock
I20260926 15:03:05.360268 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:35111
--webserver_interface=127.9.123.126
--webserver_port=0
--builtin_ntp_servers=127.9.123.84:38951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:35111
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434939095847-9709-0/testchainca.cert with env {}
W20260926 15:03:05.685343 15202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:05.685640 15202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:05.685724 15202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:05.692914 15202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:03:05.693038 15202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:05.693074 15202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:05.693156 15202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:03:05.693209 15202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260926 15:03:05.693269 15202 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260926 15:03:05.693307 15202 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260926 15:03:05.703260 15202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:38951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:35111
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:35111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434939095847-9709-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:05.704550 15202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:05.705847 15202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:05.717577 15207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:05.717602 15208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:05.717850 15210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:05.718291 15202 server_base.cc:1034] running on GCE node
I20260926 15:03:05.719255 15202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:05.720701 15202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:05.721925 15202 hybrid_clock.cc:648] HybridClock initialized: now 1790434985721859 us; error 55 us; skew 500 ppm
I20260926 15:03:05.722306 15202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:03:05.724675 15202 webserver.cc:460] Webserver started at http://127.9.123.126:46041/ using document root <none> and password file <none>
I20260926 15:03:05.725432 15202 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:05.725544 15202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:05.725841 15202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:05.728736 15202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "7a2d953fc789457b9d84a0026d4dbc0d"
format_stamp: "Formatted at 2026-09-26 15:03:05 on dist-test-slave-7153"
server_key: "5f51a749b4ef05579b8c887e57c0747f"
server_key_iv: "65e6be4adf0fe5007024489de3fe2ff6"
server_key_version: "encryptionkey@0"
I20260926 15:03:05.729557 15202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "7a2d953fc789457b9d84a0026d4dbc0d"
format_stamp: "Formatted at 2026-09-26 15:03:05 on dist-test-slave-7153"
server_key: "5f51a749b4ef05579b8c887e57c0747f"
server_key_iv: "65e6be4adf0fe5007024489de3fe2ff6"
server_key_version: "encryptionkey@0"
I20260926 15:03:05.734871 15202 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 15:03:05.738448 15216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:05.739912 15202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:03:05.740091 15202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "7a2d953fc789457b9d84a0026d4dbc0d"
format_stamp: "Formatted at 2026-09-26 15:03:05 on dist-test-slave-7153"
server_key: "5f51a749b4ef05579b8c887e57c0747f"
server_key_iv: "65e6be4adf0fe5007024489de3fe2ff6"
server_key_version: "encryptionkey@0"
I20260926 15:03:05.740296 15202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:05.752846 15202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:05.753723 15202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:05.753983 15202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:05.771538 15202 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:35111
I20260926 15:03:05.771559 15267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:35111 every 8 connection(s)
I20260926 15:03:05.772923 15202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:03:05.776845 15268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:05.781162 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 15202
I20260926 15:03:05.781348 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:03:05.782003 9709 external_mini_cluster.cc:1428] Setting key 757b8d639ec52f7db1a6a2547dea5e55
I20260926 15:03:05.788409 15268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a2d953fc789457b9d84a0026d4dbc0d: Bootstrap starting.
I20260926 15:03:05.791803 15268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a2d953fc789457b9d84a0026d4dbc0d: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:05.792968 15268 log.cc:824] T 00000000000000000000000000000000 P 7a2d953fc789457b9d84a0026d4dbc0d: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:05.796677 15268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a2d953fc789457b9d84a0026d4dbc0d: No bootstrap required, opened a new log
I20260926 15:03:05.796767 15271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:05.783960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58288 (local address 127.9.123.126:35111)
0926 15:03:05.785029 (+ 1069us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:05.785041 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:05.785085 (+ 44us) server_negotiation.cc:407] Connection header received
0926 15:03:05.786989 (+ 1904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:05.787016 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:05.787525 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:05.787916 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:05.788504 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:05.789960 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:05.791360 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:05.792191 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:05.792398 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:05.793760 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:05.793770 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:05.793787 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:05.794442 (+ 655us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:05.794550 (+ 108us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:05.794578 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:05.794714 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:05.794966 (+ 252us) server_negotiation.cc:299] Negotiation successful
0926 15:03:05.795333 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":356,"threads_started":1}
I20260926 15:03:05.801398 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:03:05.805152 15268 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a2d953fc789457b9d84a0026d4dbc0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a2d953fc789457b9d84a0026d4dbc0d" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 35111 } }
I20260926 15:03:05.805645 15268 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a2d953fc789457b9d84a0026d4dbc0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:05.805754 15268 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a2d953fc789457b9d84a0026d4dbc0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a2d953fc789457b9d84a0026d4dbc0d, State: Initialized, Role: FOLLOWER
I20260926 15:03:05.806380 15268 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a2d953fc789457b9d84a0026d4dbc0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a2d953fc789457b9d84a0026d4dbc0d" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 35111 } }
I20260926 15:03:05.806795 15268 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a2d953fc789457b9d84a0026d4dbc0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:03:05.806988 15268 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a2d953fc789457b9d84a0026d4dbc0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:03:05.807318 15268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a2d953fc789457b9d84a0026d4dbc0d [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:05.809774 15268 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a2d953fc789457b9d84a0026d4dbc0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a2d953fc789457b9d84a0026d4dbc0d" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 35111 } }
I20260926 15:03:05.810372 15268 leader_election.cc:302] T 00000000000000000000000000000000 P 7a2d953fc789457b9d84a0026d4dbc0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a2d953fc789457b9d84a0026d4dbc0d; no voters:
I20260926 15:03:05.811039 15268 leader_election.cc:288] T 00000000000000000000000000000000 P 7a2d953fc789457b9d84a0026d4dbc0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:03:05.811337 15273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a2d953fc789457b9d84a0026d4dbc0d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:03:05.812335 15273 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a2d953fc789457b9d84a0026d4dbc0d [term 1 LEADER]: Becoming Leader. State: Replica: 7a2d953fc789457b9d84a0026d4dbc0d, State: Running, Role: LEADER
I20260926 15:03:05.812958 15273 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a2d953fc789457b9d84a0026d4dbc0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a2d953fc789457b9d84a0026d4dbc0d" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 35111 } }
I20260926 15:03:05.813390 15268 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a2d953fc789457b9d84a0026d4dbc0d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:03:05.815896 15274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a2d953fc789457b9d84a0026d4dbc0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a2d953fc789457b9d84a0026d4dbc0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a2d953fc789457b9d84a0026d4dbc0d" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 35111 } } }
I20260926 15:03:05.816498 15274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a2d953fc789457b9d84a0026d4dbc0d [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:05.818560 15282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:03:05.821189 15275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a2d953fc789457b9d84a0026d4dbc0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a2d953fc789457b9d84a0026d4dbc0d. Latest consensus state: current_term: 1 leader_uuid: "7a2d953fc789457b9d84a0026d4dbc0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a2d953fc789457b9d84a0026d4dbc0d" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 35111 } } }
I20260926 15:03:05.821664 15275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a2d953fc789457b9d84a0026d4dbc0d [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:05.824254 15282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:03:05.834714 15282 catalog_manager.cc:1347] Generated new cluster ID: 88cdceccfc3d4c048b010b752f5d47fc
I20260926 15:03:05.834841 15282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:03:05.848831 15282 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:03:05.850201 15282 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:03:05.863512 15282 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a2d953fc789457b9d84a0026d4dbc0d: Generated new TSK 0
I20260926 15:03:05.864459 15282 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:03:06.368609 15271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:05.877602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58292 (local address 127.9.123.126:35111)
0926 15:03:05.877753 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:05.877759 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:05.878083 (+ 324us) server_negotiation.cc:407] Connection header received
0926 15:03:05.878301 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:05.878307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:05.878412 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:05.878591 (+ 179us) server_negotiation.cc:226] Negotiated authn=JWT
0926 15:03:05.879010 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:05.880135 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:05.881738 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:05.882424 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:05.882560 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:05.883084 (+ 524us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0926 15:03:06.368022 (+484938us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 15:03:06.368367 (+ 345us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58292: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":42}
W20260926 15:03:06.368716 15271 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58292: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260926 15:03:06.368973 15297 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:05.877401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:35111 (local address 127.0.0.1:58292)
0926 15:03:05.877975 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:05.877991 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:05.878119 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:05.878606 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:05.878609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:05.878618 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0926 15:03:05.878875 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:05.878882 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:05.880320 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:05.880325 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:05.881581 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:05.881596 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:05.882613 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:05.882618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:05.882752 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:05.882788 (+ 36us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0926 15:03:06.368331 (+485543us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0926 15:03:06.368601 (+ 270us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.123.126:35111: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":297,"threads_started":1}
I20260926 15:03:06.370101 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 15202
I20260926 15:03:06.371178 15263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:06.470329 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 15202
2026-09-26T15:03:06Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1167 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (406 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (405 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:03:07 dist-test-slave-7153 krb5kdc[15312](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:03:07 dist-test-slave-7153 krb5kdc[15312](info): set up 2 sockets
Sep 26 15:03:07 dist-test-slave-7153 krb5kdc[15312](info): commencing operation
krb5kdc: starting...
W20260926 15:03:08.297533 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.981s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:03:08 dist-test-slave-7153 krb5kdc[15312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:03:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:03:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:03:08.498420 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:40579
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:38245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:40579
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:03:08.818823 15325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:08.819151 15325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:08.819272 15325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:08.826467 15325 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:03:08.826587 15325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:08.826661 15325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:08.826732 15325 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:03:08.826790 15325 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:03:08.836534 15325 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:38245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:40579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:40579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:08.837852 15325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:08.839123 15325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:08.850286 15330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:08.850278 15333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:08.850274 15331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:08.851377 15325 server_base.cc:1034] running on GCE node
I20260926 15:03:08.852183 15325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:08.853772 15325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:08.854979 15325 hybrid_clock.cc:648] HybridClock initialized: now 1790434988854926 us; error 49 us; skew 500 ppm
I20260926 15:03:08.855322 15325 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:08 dist-test-slave-7153 krb5kdc[15312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434988, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:08.858321 15325 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:03:08.859745 15325 webserver.cc:460] Webserver started at http://127.9.123.126:41523/ using document root <none> and password file <none>
I20260926 15:03:08.860414 15325 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:08.860528 15325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:08.860826 15325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:08.863889 15325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "df4852252df4467f9f5a8ed99bc0f5e6"
format_stamp: "Formatted at 2026-09-26 15:03:08 on dist-test-slave-7153"
server_key: "21615ba7baa9e3744f4c671db4c56320"
server_key_iv: "4bacc9f05dfed0d80a6357f2858950cf"
server_key_version: "encryptionkey@0"
I20260926 15:03:08.864686 15325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "df4852252df4467f9f5a8ed99bc0f5e6"
format_stamp: "Formatted at 2026-09-26 15:03:08 on dist-test-slave-7153"
server_key: "21615ba7baa9e3744f4c671db4c56320"
server_key_iv: "4bacc9f05dfed0d80a6357f2858950cf"
server_key_version: "encryptionkey@0"
I20260926 15:03:08.870479 15325 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.004s
I20260926 15:03:08.874235 15340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:08.875977 15325 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:03:08.876163 15325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "df4852252df4467f9f5a8ed99bc0f5e6"
format_stamp: "Formatted at 2026-09-26 15:03:08 on dist-test-slave-7153"
server_key: "21615ba7baa9e3744f4c671db4c56320"
server_key_iv: "4bacc9f05dfed0d80a6357f2858950cf"
server_key_version: "encryptionkey@0"
I20260926 15:03:08.876367 15325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:08.902094 15325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:08.906131 15325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:08.906395 15325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:08.923204 15325 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:40579
I20260926 15:03:08.923249 15391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:40579 every 8 connection(s)
I20260926 15:03:08.924511 15325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:03:08.927073 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 15325
I20260926 15:03:08.927278 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:03:08.928062 9709 external_mini_cluster.cc:1428] Setting key 0b4b718d9083c95e65664d379eef490a
I20260926 15:03:08.928916 15392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 15:03:08 dist-test-slave-7153 krb5kdc[15312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:03:08.943290 15392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df4852252df4467f9f5a8ed99bc0f5e6: Bootstrap starting.
I20260926 15:03:08.946551 15392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df4852252df4467f9f5a8ed99bc0f5e6: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:08.947889 15392 log.cc:824] T 00000000000000000000000000000000 P df4852252df4467f9f5a8ed99bc0f5e6: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:08.950721 15395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:08.930085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53588 (local address 127.9.123.126:40579)
0926 15:03:08.930809 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:08.930821 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:08.930861 (+ 40us) server_negotiation.cc:407] Connection header received
0926 15:03:08.932310 (+ 1449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:08.932333 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:08.932741 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:08.933087 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:08.934814 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:08.936167 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:08.937833 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:08.938535 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:08.938672 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:08.943455 (+ 4783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:08.943464 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:08.943481 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:08.943519 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:08.946366 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:08.947273 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:08.947281 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:08.947287 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:08.947463 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:08.947911 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:08.947915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:08.947917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:08.948601 (+ 684us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:08.948904 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:08.949282 (+ 378us) server_negotiation.cc:299] Negotiation successful
0926 15:03:08.949685 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":303,"threads_started":1}
I20260926 15:03:08.950968 15392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df4852252df4467f9f5a8ed99bc0f5e6: No bootstrap required, opened a new log
I20260926 15:03:08.960249 15392 raft_consensus.cc:348] T 00000000000000000000000000000000 P df4852252df4467f9f5a8ed99bc0f5e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df4852252df4467f9f5a8ed99bc0f5e6" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 40579 } }
I20260926 15:03:08.960757 15392 raft_consensus.cc:374] T 00000000000000000000000000000000 P df4852252df4467f9f5a8ed99bc0f5e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:08.960860 15392 raft_consensus.cc:729] T 00000000000000000000000000000000 P df4852252df4467f9f5a8ed99bc0f5e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df4852252df4467f9f5a8ed99bc0f5e6, State: Initialized, Role: FOLLOWER
I20260926 15:03:08.961602 15392 consensus_queue.cc:260] T 00000000000000000000000000000000 P df4852252df4467f9f5a8ed99bc0f5e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df4852252df4467f9f5a8ed99bc0f5e6" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 40579 } }
I20260926 15:03:08.961926 15392 raft_consensus.cc:388] T 00000000000000000000000000000000 P df4852252df4467f9f5a8ed99bc0f5e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:03:08.962083 15392 raft_consensus.cc:482] T 00000000000000000000000000000000 P df4852252df4467f9f5a8ed99bc0f5e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:03:08.962271 15392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df4852252df4467f9f5a8ed99bc0f5e6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:08.965139 15392 raft_consensus.cc:504] T 00000000000000000000000000000000 P df4852252df4467f9f5a8ed99bc0f5e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df4852252df4467f9f5a8ed99bc0f5e6" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 40579 } }
I20260926 15:03:08.965618 15392 leader_election.cc:302] T 00000000000000000000000000000000 P df4852252df4467f9f5a8ed99bc0f5e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df4852252df4467f9f5a8ed99bc0f5e6; no voters:
I20260926 15:03:08.966106 15392 leader_election.cc:288] T 00000000000000000000000000000000 P df4852252df4467f9f5a8ed99bc0f5e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:03:08.966228 15397 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df4852252df4467f9f5a8ed99bc0f5e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:03:08.966821 15397 raft_consensus.cc:686] T 00000000000000000000000000000000 P df4852252df4467f9f5a8ed99bc0f5e6 [term 1 LEADER]: Becoming Leader. State: Replica: df4852252df4467f9f5a8ed99bc0f5e6, State: Running, Role: LEADER
I20260926 15:03:08.967373 15397 consensus_queue.cc:237] T 00000000000000000000000000000000 P df4852252df4467f9f5a8ed99bc0f5e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df4852252df4467f9f5a8ed99bc0f5e6" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 40579 } }
I20260926 15:03:08.968123 15392 sys_catalog.cc:564] T 00000000000000000000000000000000 P df4852252df4467f9f5a8ed99bc0f5e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:03:08.970094 15399 sys_catalog.cc:455] T 00000000000000000000000000000000 P df4852252df4467f9f5a8ed99bc0f5e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df4852252df4467f9f5a8ed99bc0f5e6. Latest consensus state: current_term: 1 leader_uuid: "df4852252df4467f9f5a8ed99bc0f5e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df4852252df4467f9f5a8ed99bc0f5e6" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 40579 } } }
I20260926 15:03:08.970466 15399 sys_catalog.cc:458] T 00000000000000000000000000000000 P df4852252df4467f9f5a8ed99bc0f5e6 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:08.971619 15398 sys_catalog.cc:455] T 00000000000000000000000000000000 P df4852252df4467f9f5a8ed99bc0f5e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df4852252df4467f9f5a8ed99bc0f5e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df4852252df4467f9f5a8ed99bc0f5e6" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 40579 } } }
I20260926 15:03:08.971962 15398 sys_catalog.cc:458] T 00000000000000000000000000000000 P df4852252df4467f9f5a8ed99bc0f5e6 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:08.973074 15402 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:03:08.977277 15402 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:03:08.985530 15402 catalog_manager.cc:1347] Generated new cluster ID: 50178997f189496c91d2886f264152a0
I20260926 15:03:08.985706 15402 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:03:08.999214 15402 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:03:09.001058 15402 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:03:09.011715 15402 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df4852252df4467f9f5a8ed99bc0f5e6: Generated new TSK 0
I20260926 15:03:09.012826 15402 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.123.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
I20260926 15:03:09.061313 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:40579
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:38245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:03:09.389966 15418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:09.390271 15418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:09.390365 15418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:09.397815 15418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:09.397948 15418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:09.398072 15418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:03:09.408172 15418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:38245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:40579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:09.409490 15418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:09.410936 15418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:09.424433 15423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:09.424358 15426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:09.424314 15424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:09.425076 15418 server_base.cc:1034] running on GCE node
I20260926 15:03:09.425807 15418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:09.426826 15418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:09.428036 15418 hybrid_clock.cc:648] HybridClock initialized: now 1790434989427974 us; error 56 us; skew 500 ppm
I20260926 15:03:09.428366 15418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:09 dist-test-slave-7153 krb5kdc[15312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434989, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:09.431690 15418 init.cc:377] Logged in from keytab as kudu/127.9.123.65@KRBTEST.COM (short username kudu)
I20260926 15:03:09.433241 15418 webserver.cc:460] Webserver started at http://127.9.123.65:35069/ using document root <none> and password file <none>
I20260926 15:03:09.433977 15418 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:09.434108 15418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:09.434422 15418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:09.437594 15418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "b8f4119a0e444f01bbbf4b953558390d"
format_stamp: "Formatted at 2026-09-26 15:03:09 on dist-test-slave-7153"
server_key: "b306c58f3f240487f5d871177f788fdf"
server_key_iv: "12d7bdbe19171f3dc92e01be4b3b2e5c"
server_key_version: "encryptionkey@0"
I20260926 15:03:09.438426 15418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "b8f4119a0e444f01bbbf4b953558390d"
format_stamp: "Formatted at 2026-09-26 15:03:09 on dist-test-slave-7153"
server_key: "b306c58f3f240487f5d871177f788fdf"
server_key_iv: "12d7bdbe19171f3dc92e01be4b3b2e5c"
server_key_version: "encryptionkey@0"
I20260926 15:03:09.444036 15418 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 15:03:09.447935 15433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:09.449622 15418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:03:09.449836 15418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "b8f4119a0e444f01bbbf4b953558390d"
format_stamp: "Formatted at 2026-09-26 15:03:09 on dist-test-slave-7153"
server_key: "b306c58f3f240487f5d871177f788fdf"
server_key_iv: "12d7bdbe19171f3dc92e01be4b3b2e5c"
server_key_version: "encryptionkey@0"
I20260926 15:03:09.450091 15418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:09.472117 15418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:09.476347 15418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:09.476601 15418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:09.477619 15418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:09.478900 15418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:09.478978 15418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:09.479080 15418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:09.479142 15418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:09.508827 15418 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:44575
I20260926 15:03:09.508864 15545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:44575 every 8 connection(s)
I20260926 15:03:09.510143 15418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:03:09.510828 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 15418
I20260926 15:03:09.511022 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:03:09.511831 9709 external_mini_cluster.cc:1428] Setting key 992cefa5150e2eaddff25b3d5552a5f5
Sep 26 15:03:09 dist-test-slave-7153 krb5kdc[15312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434989, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.123.66@KRBTEST.COM; defaulting to no policy
I20260926 15:03:09.534876 15548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:09.513088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:56353 (local address 127.9.123.126:40579)
0926 15:03:09.513565 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:09.513571 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:09.514743 (+ 1172us) server_negotiation.cc:407] Connection header received
0926 15:03:09.517221 (+ 2478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:09.517225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:09.517300 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:09.517382 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:09.519441 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:09.520336 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:09.522124 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:09.522739 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:09.522867 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:09.527677 (+ 4810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:09.527683 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:09.527686 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:09.527724 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:09.531156 (+ 3432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:09.532266 (+ 1110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:09.532273 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:09.532277 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:09.532446 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:09.533054 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:09.533060 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:09.533063 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:09.533388 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:09.533496 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:09.534305 (+ 809us) server_negotiation.cc:299] Negotiation successful
0926 15:03:09.534531 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":219,"threads_started":1}
I20260926 15:03:09.536015 15549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:09.513755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:40579 (local address 127.9.123.65:56353)
0926 15:03:09.514583 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:09.514633 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:09.516850 (+ 2217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:09.517665 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:09.517678 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:09.518398 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:09.519258 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:09.519284 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:09.520512 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:09.520517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:09.521939 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:09.521953 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:09.523023 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:09.523032 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:09.523364 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:09.524190 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:09.524223 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:09.527314 (+ 3091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:09.531388 (+ 4074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:09.531400 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:09.531423 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:09.532099 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:09.532627 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:09.532633 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:09.532636 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:09.532919 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:09.533572 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:09.533579 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:09.533929 (+ 350us) client_negotiation.cc:769] Sending connection context
0926 15:03:09.534174 (+ 245us) client_negotiation.cc:240] Negotiation successful
0926 15:03:09.534757 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":333,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.66@KRBTEST.COM" created.
I20260926 15:03:09.537751 15546 heartbeater.cc:347] Connected to a master server at 127.9.123.126:40579
I20260926 15:03:09.538095 15546 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:09.539016 15546 heartbeater.cc:511] Master 127.9.123.126:40579 requested a full tablet report, sending...
I20260926 15:03:09.541499 15357 ts_manager.cc:194] Registered new tserver with Master: b8f4119a0e444f01bbbf4b953558390d (127.9.123.65:44575)
I20260926 15:03:09.543368 15357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.65@KRBTEST.COM'} at 127.9.123.65:56353
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
I20260926 15:03:09.564364 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:40579
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:38245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:03:09.896250 15552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:09.896554 15552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:09.896647 15552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:09.904054 15552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:09.904179 15552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:09.904292 15552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:03:09.914383 15552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:38245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:40579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:09.915714 15552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:09.916987 15552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:09.931062 15560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:09.931041 15558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:09.931593 15557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:09.939082 15552 server_base.cc:1034] running on GCE node
I20260926 15:03:09.939795 15552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:09.940819 15552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:09.942068 15552 hybrid_clock.cc:648] HybridClock initialized: now 1790434989941992 us; error 66 us; skew 500 ppm
I20260926 15:03:09.942431 15552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:09 dist-test-slave-7153 krb5kdc[15312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434989, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:09.945698 15552 init.cc:377] Logged in from keytab as kudu/127.9.123.66@KRBTEST.COM (short username kudu)
I20260926 15:03:09.947353 15552 webserver.cc:460] Webserver started at http://127.9.123.66:44151/ using document root <none> and password file <none>
I20260926 15:03:09.948071 15552 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:09.948194 15552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:09.948475 15552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:09.951543 15552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "fef49e8a4f564d1a8b1d2c549905fe2f"
format_stamp: "Formatted at 2026-09-26 15:03:09 on dist-test-slave-7153"
server_key: "5a33ebd065796d290b161ba5db9cf5a2"
server_key_iv: "27389862c29831a0d2c24173f5611bf7"
server_key_version: "encryptionkey@0"
I20260926 15:03:09.952380 15552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "fef49e8a4f564d1a8b1d2c549905fe2f"
format_stamp: "Formatted at 2026-09-26 15:03:09 on dist-test-slave-7153"
server_key: "5a33ebd065796d290b161ba5db9cf5a2"
server_key_iv: "27389862c29831a0d2c24173f5611bf7"
server_key_version: "encryptionkey@0"
I20260926 15:03:09.958050 15552 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260926 15:03:09.962116 15567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:09.963748 15552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:03:09.963946 15552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "fef49e8a4f564d1a8b1d2c549905fe2f"
format_stamp: "Formatted at 2026-09-26 15:03:09 on dist-test-slave-7153"
server_key: "5a33ebd065796d290b161ba5db9cf5a2"
server_key_iv: "27389862c29831a0d2c24173f5611bf7"
server_key_version: "encryptionkey@0"
I20260926 15:03:09.964183 15552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:09.999547 15552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:10.003749 15552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:10.003999 15552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:10.004997 15552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:10.006474 15552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:10.006551 15552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:10.006659 15552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:10.006714 15552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:10.039106 15552 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:35735
I20260926 15:03:10.039160 15679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:35735 every 8 connection(s)
I20260926 15:03:10.040452 15552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:03:10.045771 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 15552
I20260926 15:03:10.045928 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:03:10.046368 9709 external_mini_cluster.cc:1428] Setting key 7019c1fa4f534703213c318ff1b6df88
Sep 26 15:03:10 dist-test-slave-7153 krb5kdc[15312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434989, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:03:10.061753 15682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:10.043146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:59657 (local address 127.9.123.126:40579)
0926 15:03:10.043654 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:10.043660 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:10.044767 (+ 1107us) server_negotiation.cc:407] Connection header received
0926 15:03:10.046849 (+ 2082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:10.046854 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:10.046961 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:10.047061 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:10.049011 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:10.049797 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:10.051466 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:10.051910 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:10.052023 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:10.056064 (+ 4041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:10.056070 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:10.056073 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:10.056109 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:10.058610 (+ 2501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:10.059491 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:10.059494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:10.059497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:10.059641 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:10.060082 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:10.060085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:10.060087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:10.060328 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:10.060428 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:10.061314 (+ 886us) server_negotiation.cc:299] Negotiation successful
0926 15:03:10.061481 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":235,"threads_started":1}
I20260926 15:03:10.063354 15683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:10.043789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:40579 (local address 127.9.123.66:59657)
0926 15:03:10.044579 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:10.044625 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:10.046509 (+ 1884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:10.047390 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:10.047403 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:10.048186 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:10.048826 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:10.048847 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:10.049961 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:10.049965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:10.051307 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:10.051320 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:10.052161 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:10.052169 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:10.052404 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:10.053080 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:10.053136 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:10.055788 (+ 2652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:10.058762 (+ 2974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:10.058772 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:10.058790 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:10.059385 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:10.059736 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:10.059742 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:10.059745 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:10.059989 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:10.060435 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:10.060442 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:10.060894 (+ 452us) client_negotiation.cc:769] Sending connection context
0926 15:03:10.061254 (+ 360us) client_negotiation.cc:240] Negotiation successful
0926 15:03:10.061862 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":326,"threads_started":1}
I20260926 15:03:10.065003 15680 heartbeater.cc:347] Connected to a master server at 127.9.123.126:40579
WARNING: no policy specified for kudu/127.9.123.67@KRBTEST.COM; defaulting to no policy
I20260926 15:03:10.065347 15680 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:10.066075 15680 heartbeater.cc:511] Master 127.9.123.126:40579 requested a full tablet report, sending...
I20260926 15:03:10.067576 15357 ts_manager.cc:194] Registered new tserver with Master: fef49e8a4f564d1a8b1d2c549905fe2f (127.9.123.66:35735)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.67@KRBTEST.COM" created.
I20260926 15:03:10.068567 15357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.66@KRBTEST.COM'} at 127.9.123.66:59657
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
I20260926 15:03:10.092547 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:40579
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:38245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:03:10.425804 15686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:10.426098 15686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:10.426190 15686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:10.433589 15686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:10.433717 15686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:10.433831 15686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:03:10.443780 15686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:38245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:40579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:10.445052 15686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:10.446388 15686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:10.459693 15692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:10.459662 15694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:10.459929 15691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:10.460423 15686 server_base.cc:1034] running on GCE node
I20260926 15:03:10.461124 15686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:10.462100 15686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:10.463322 15686 hybrid_clock.cc:648] HybridClock initialized: now 1790434990463265 us; error 50 us; skew 500 ppm
I20260926 15:03:10.463676 15686 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:10 dist-test-slave-7153 krb5kdc[15312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434990, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:10.466837 15686 init.cc:377] Logged in from keytab as kudu/127.9.123.67@KRBTEST.COM (short username kudu)
I20260926 15:03:10.468493 15686 webserver.cc:460] Webserver started at http://127.9.123.67:45145/ using document root <none> and password file <none>
I20260926 15:03:10.469269 15686 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:10.469375 15686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:10.469645 15686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:10.472589 15686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "446b8ffb280843d6b7b5b855f38ef817"
format_stamp: "Formatted at 2026-09-26 15:03:10 on dist-test-slave-7153"
server_key: "8402053f92025295882f6d89cfc656a2"
server_key_iv: "d00cb17eb1e06f466072db358890e770"
server_key_version: "encryptionkey@0"
I20260926 15:03:10.473371 15686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "446b8ffb280843d6b7b5b855f38ef817"
format_stamp: "Formatted at 2026-09-26 15:03:10 on dist-test-slave-7153"
server_key: "8402053f92025295882f6d89cfc656a2"
server_key_iv: "d00cb17eb1e06f466072db358890e770"
server_key_version: "encryptionkey@0"
I20260926 15:03:10.478766 15686 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260926 15:03:10.482779 15701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:10.484385 15686 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260926 15:03:10.484553 15686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "446b8ffb280843d6b7b5b855f38ef817"
format_stamp: "Formatted at 2026-09-26 15:03:10 on dist-test-slave-7153"
server_key: "8402053f92025295882f6d89cfc656a2"
server_key_iv: "d00cb17eb1e06f466072db358890e770"
server_key_version: "encryptionkey@0"
I20260926 15:03:10.484763 15686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:10.503638 15686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:10.508306 15686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:10.508656 15686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:10.509758 15686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:10.511176 15686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:10.511257 15686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:10.511377 15686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:10.511428 15686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:10.541908 15686 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:35877
I20260926 15:03:10.541947 15813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:35877 every 8 connection(s)
I20260926 15:03:10.543200 15686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:03:10.547098 15546 heartbeater.cc:503] Master 127.9.123.126:40579 was elected leader, sending a full tablet report...
I20260926 15:03:10.552959 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 15686
I20260926 15:03:10.553133 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:03:10.553732 9709 external_mini_cluster.cc:1428] Setting key ae282f15b82878bfa20547a3e5ec7c88
Sep 26 15:03:10 dist-test-slave-7153 krb5kdc[15312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434990, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:03:10.562285 15682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:10.545727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:58169 (local address 127.9.123.126:40579)
0926 15:03:10.545915 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:10.545921 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:10.547210 (+ 1289us) server_negotiation.cc:407] Connection header received
0926 15:03:10.549180 (+ 1970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:10.549185 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:10.549279 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:10.549362 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:10.550892 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:10.551601 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:10.553193 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:10.553782 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:10.553913 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:10.557304 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:10.557309 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:10.557312 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:10.557344 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:10.559610 (+ 2266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:10.560355 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:10.560359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:10.560361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:10.560478 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:10.560811 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:10.560813 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:10.560815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:10.561066 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:10.561241 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:10.561896 (+ 655us) server_negotiation.cc:299] Negotiation successful
0926 15:03:10.562063 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 15:03:10.563335 15816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:10.546275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:40579 (local address 127.9.123.67:58169)
0926 15:03:10.547023 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:10.547078 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:10.548896 (+ 1818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:10.549618 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:10.549626 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:10.550145 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:10.550726 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:10.550744 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:10.551745 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:10.551749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:10.552993 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:10.553006 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:10.553999 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:10.554004 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:10.554230 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:10.554901 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:10.554939 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:10.557062 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:10.559775 (+ 2713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:10.559783 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:10.559797 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:10.560262 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:10.560550 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:10.560553 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:10.560555 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:10.560727 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:10.561236 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:10.561243 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:10.561585 (+ 342us) client_negotiation.cc:769] Sending connection context
0926 15:03:10.561820 (+ 235us) client_negotiation.cc:240] Negotiation successful
0926 15:03:10.562229 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":289,"threads_started":1}
I20260926 15:03:10.564690 15814 heartbeater.cc:347] Connected to a master server at 127.9.123.126:40579
I20260926 15:03:10.564983 15814 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:10.565771 15814 heartbeater.cc:511] Master 127.9.123.126:40579 requested a full tablet report, sending...
I20260926 15:03:10.567346 15357 ts_manager.cc:194] Registered new tserver with Master: 446b8ffb280843d6b7b5b855f38ef817 (127.9.123.67:35877)
I20260926 15:03:10.568339 15357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.67@KRBTEST.COM'} at 127.9.123.67:58169
I20260926 15:03:10.569648 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/test-admin.keytab.
Sep 26 15:03:10 dist-test-slave-7153 krb5kdc[15312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:10.599881 9709 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260926 15:03:10.605399 15821 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:10.602768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43140 (local address 127.9.123.65:44575)
0926 15:03:10.603477 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:10.603489 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:10.603519 (+ 30us) server_negotiation.cc:407] Connection header received
0926 15:03:10.603580 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:10.603586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:10.603818 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:10.603970 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:10.604945 (+ 975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43140: BlockingRecv error: recv got EOF from 127.0.0.1:43140 (error 108)
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":413,"threads_started":1}
W20260926 15:03:10.606016 9709 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/krb5cc' of type FILE
Sep 26 15:03:10 dist-test-slave-7153 krb5kdc[15312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:10.607772 9709 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 15:03:10 dist-test-slave-7153 krb5kdc[15312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:10.609318 9709 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 15:03:10 dist-test-slave-7153 krb5kdc[15312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.65@KRBTEST.COM
I20260926 15:03:10.626639 15821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:10.611200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43156 (local address 127.9.123.65:44575)
0926 15:03:10.611412 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:10.611417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:10.611482 (+ 65us) server_negotiation.cc:407] Connection header received
0926 15:03:10.611663 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:10.611666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:10.611726 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:10.611812 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:10.612895 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:10.613989 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:10.615457 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:10.616123 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:10.616286 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:10.620350 (+ 4064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:10.620366 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:10.620382 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:10.620430 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:10.623509 (+ 3079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:10.624379 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:10.624388 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:10.624391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:10.624545 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:10.624998 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:10.625003 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:10.625005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:10.625491 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:10.625734 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:10.626061 (+ 327us) server_negotiation.cc:299] Negotiation successful
0926 15:03:10.626338 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260926 15:03:10.632982 15821 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:10.631137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43158 (local address 127.9.123.65:44575)
0926 15:03:10.631292 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:10.631296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:10.631621 (+ 325us) server_negotiation.cc:407] Connection header received
0926 15:03:10.631760 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:10.631763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:10.631823 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:10.631905 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:10.632734 (+ 829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43158: BlockingRecv error: recv got EOF from 127.0.0.1:43158 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260926 15:03:10.633296 9709 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/krb5cc' of type FILE
Sep 26 15:03:10 dist-test-slave-7153 krb5kdc[15312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:10.634770 9709 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 15:03:10 dist-test-slave-7153 krb5kdc[15312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:10.636077 9709 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 15:03:10 dist-test-slave-7153 krb5kdc[15312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.65@KRBTEST.COM
I20260926 15:03:10.651633 15821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:10.637741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43168 (local address 127.9.123.65:44575)
0926 15:03:10.637896 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:10.637901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:10.638095 (+ 194us) server_negotiation.cc:407] Connection header received
0926 15:03:10.638259 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:10.638263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:10.638339 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:10.638429 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:10.639458 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:10.640188 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:10.641601 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:10.642072 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:10.642193 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:10.646008 (+ 3815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:10.646014 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:10.646017 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:10.646074 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:10.648745 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:10.649594 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:10.649599 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:10.649601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:10.649751 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:10.650316 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:10.650327 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:10.650331 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:10.650640 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:10.650847 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:10.651211 (+ 364us) server_negotiation.cc:299] Negotiation successful
0926 15:03:10.651392 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260926 15:03:10.656332 15821 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:10.654546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43182 (local address 127.9.123.65:44575)
0926 15:03:10.654745 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:10.654751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:10.654919 (+ 168us) server_negotiation.cc:407] Connection header received
0926 15:03:10.655128 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:10.655133 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:10.655208 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:10.655293 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:10.656098 (+ 805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43182: BlockingRecv error: recv got EOF from 127.0.0.1:43182 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260926 15:03:10.656487 9709 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790434939095847-9709-0/krb5kdc/krb5cc' of type FILE
Sep 26 15:03:10 dist-test-slave-7153 krb5kdc[15312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:10.658149 9709 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 26 15:03:10 dist-test-slave-7153 krb5kdc[15312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:10.659682 9709 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 26 15:03:10 dist-test-slave-7153 krb5kdc[15312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.65@KRBTEST.COM
I20260926 15:03:10.676584 15821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:10.661407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43198 (local address 127.9.123.65:44575)
0926 15:03:10.661626 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:10.661632 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:10.661841 (+ 209us) server_negotiation.cc:407] Connection header received
0926 15:03:10.662039 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:10.662043 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:10.662118 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:10.662226 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:10.663264 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:10.663997 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:10.665518 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:10.665966 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:10.666089 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:10.670565 (+ 4476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:10.670572 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:10.670575 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:10.670617 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:10.673462 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:10.674346 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:10.674351 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:10.674354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:10.674555 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:10.675069 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:10.675074 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:10.675076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:10.675390 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:10.675635 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:10.676037 (+ 402us) server_negotiation.cc:299] Negotiation successful
0926 15:03:10.676276 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260926 15:03:10.678328 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 15418
I20260926 15:03:10.691988 15821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:10.679092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43210 (local address 127.9.123.65:44575)
0926 15:03:10.679247 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:10.679251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:10.679619 (+ 368us) server_negotiation.cc:407] Connection header received
0926 15:03:10.679784 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:10.679789 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:10.679861 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:10.679954 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:10.680915 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:10.681865 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:10.683677 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:10.684142 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:10.684284 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:10.686518 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:10.686523 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:10.686526 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:10.686573 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:10.689287 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:10.690014 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:10.690020 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:10.690022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:10.690175 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:10.690580 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:10.690584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:10.690586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:10.690908 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:10.691167 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:10.691454 (+ 287us) server_negotiation.cc:299] Negotiation successful
0926 15:03:10.691665 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 15:03:10.692565 15541 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:10.780474 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 15418
I20260926 15:03:10.794106 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 15552
Sep 26 15:03:10 dist-test-slave-7153 krb5kdc[15312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.66@KRBTEST.COM
I20260926 15:03:10.811477 15834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:10.795077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37124 (local address 127.9.123.66:35735)
0926 15:03:10.795593 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:10.795600 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:10.795627 (+ 27us) server_negotiation.cc:407] Connection header received
0926 15:03:10.795712 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:10.795718 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:10.795934 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:10.796062 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:10.797074 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:10.798369 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:10.799734 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:10.800366 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:10.800474 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:10.804737 (+ 4263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:10.804754 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:10.804771 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:10.804824 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:10.807933 (+ 3109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:10.808835 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:10.808844 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:10.808846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:10.808987 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:10.809682 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:10.809687 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:10.809689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:10.810117 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:10.810369 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:10.810800 (+ 431us) server_negotiation.cc:299] Negotiation successful
0926 15:03:10.811125 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":254,"threads_started":1}
I20260926 15:03:10.812928 15675 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:10.895113 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 15552
I20260926 15:03:10.907774 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 15686
Sep 26 15:03:10 dist-test-slave-7153 krb5kdc[15312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.67@KRBTEST.COM
I20260926 15:03:10.924228 15836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:10.908862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60298 (local address 127.9.123.67:35877)
0926 15:03:10.909573 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:10.909581 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:10.909612 (+ 31us) server_negotiation.cc:407] Connection header received
0926 15:03:10.909679 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:10.909685 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:10.909922 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:10.910071 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:10.911252 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:10.912291 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:10.913718 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:10.914342 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:10.914452 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:10.918461 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:10.918477 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:10.918500 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:10.918546 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:10.921378 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:10.922098 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:10.922105 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:10.922107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:10.922236 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:10.922681 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:10.922685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:10.922687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:10.923064 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:10.923356 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:10.923658 (+ 302us) server_negotiation.cc:299] Negotiation successful
0926 15:03:10.923932 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":412,"threads_started":1}
I20260926 15:03:10.925558 15809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:11.008132 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 15686
I20260926 15:03:11.020332 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 15325
I20260926 15:03:11.021610 15387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:11.122044 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 15325
2026-09-26T15:03:11Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3843 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:03:11 dist-test-slave-7153 krb5kdc[15841](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:03:11 dist-test-slave-7153 krb5kdc[15841](info): set up 2 sockets
Sep 26 15:03:11 dist-test-slave-7153 krb5kdc[15841](info): commencing operation
krb5kdc: starting...
W20260926 15:03:12.153704 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.994s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:03:12 dist-test-slave-7153 krb5kdc[15841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:03:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:03:12Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.126.keytab.
Entry for principal oryx/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.126.keytab.
I20260926 15:03:12.355958 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:32823
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.126.keytab
--principal=oryx/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:41353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:32823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:03:12.674822 15854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:12.675110 15854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:12.675199 15854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:12.682279 15854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:03:12.682391 15854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:12.682435 15854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:12.682490 15854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:03:12.682533 15854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:03:12.692292 15854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:41353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:32823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.126.keytab
--principal=oryx/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:32823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:12.693675 15854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:12.695045 15854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:12.706611 15859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:12.706674 15862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:12.706646 15860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:12.707552 15854 server_base.cc:1034] running on GCE node
I20260926 15:03:12.708390 15854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:12.709878 15854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:12.711092 15854 hybrid_clock.cc:648] HybridClock initialized: now 1790434992711041 us; error 43 us; skew 500 ppm
I20260926 15:03:12.711443 15854 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:12 dist-test-slave-7153 krb5kdc[15841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434992, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:12.714555 15854 init.cc:377] Logged in from keytab as oryx/127.9.123.126@KRBTEST.COM (short username oryx)
I20260926 15:03:12.716068 15854 webserver.cc:460] Webserver started at http://127.9.123.126:41027/ using document root <none> and password file <none>
I20260926 15:03:12.716748 15854 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:12.716866 15854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:12.717195 15854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:12.720196 15854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "c93ba0d1fefe4605b9d7f761e1e06f96"
format_stamp: "Formatted at 2026-09-26 15:03:12 on dist-test-slave-7153"
server_key: "acf1c0af81f3b15197750d8877277edf"
server_key_iv: "41913c107c4c8eaea8f843dfaee0bf1f"
server_key_version: "encryptionkey@0"
I20260926 15:03:12.721009 15854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "c93ba0d1fefe4605b9d7f761e1e06f96"
format_stamp: "Formatted at 2026-09-26 15:03:12 on dist-test-slave-7153"
server_key: "acf1c0af81f3b15197750d8877277edf"
server_key_iv: "41913c107c4c8eaea8f843dfaee0bf1f"
server_key_version: "encryptionkey@0"
I20260926 15:03:12.726284 15854 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 15:03:12.729843 15869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:12.731195 15854 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:03:12.731376 15854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "c93ba0d1fefe4605b9d7f761e1e06f96"
format_stamp: "Formatted at 2026-09-26 15:03:12 on dist-test-slave-7153"
server_key: "acf1c0af81f3b15197750d8877277edf"
server_key_iv: "41913c107c4c8eaea8f843dfaee0bf1f"
server_key_version: "encryptionkey@0"
I20260926 15:03:12.731551 15854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:12.748456 15854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:12.752398 15854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:12.752674 15854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:12.769819 15854 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:32823
I20260926 15:03:12.769840 15920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:32823 every 8 connection(s)
I20260926 15:03:12.771124 15854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:03:12.774763 15921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:12.775038 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 15854
I20260926 15:03:12.775238 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:03:12.776031 9709 external_mini_cluster.cc:1428] Setting key 86dbea85abd99b7bbd5f27a25d0d54f5
I20260926 15:03:12.788790 15921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96: Bootstrap starting.
Sep 26 15:03:12 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM
I20260926 15:03:12.792573 15921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:12.793799 15921 log.cc:824] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:12.797243 15921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96: No bootstrap required, opened a new log
I20260926 15:03:12.799474 15924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:12.778067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49242 (local address 127.9.123.126:32823)
0926 15:03:12.778933 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:12.778948 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:12.778990 (+ 42us) server_negotiation.cc:407] Connection header received
0926 15:03:12.781040 (+ 2050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:12.781068 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:12.781595 (+ 527us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:12.781966 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:12.783740 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:12.785047 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:12.786776 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:12.787449 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:12.787578 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:12.792146 (+ 4568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:12.792154 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:12.792170 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:12.792210 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:12.795447 (+ 3237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:12.796224 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:12.796230 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:12.796236 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:12.796390 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:12.796793 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:12.796796 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:12.796798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:12.797349 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:12.797625 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:12.798050 (+ 425us) server_negotiation.cc:299] Negotiation successful
0926 15:03:12.798450 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":425,"threads_started":1}
I20260926 15:03:12.805193 15921 raft_consensus.cc:348] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } }
I20260926 15:03:12.805598 15921 raft_consensus.cc:374] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:12.805696 15921 raft_consensus.cc:729] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93ba0d1fefe4605b9d7f761e1e06f96, State: Initialized, Role: FOLLOWER
I20260926 15:03:12.806371 15921 consensus_queue.cc:260] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } }
I20260926 15:03:12.806592 15921 raft_consensus.cc:388] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:03:12.806725 15921 raft_consensus.cc:482] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:03:12.806893 15921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:12.809048 15921 raft_consensus.cc:504] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } }
I20260926 15:03:12.809556 15921 leader_election.cc:302] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c93ba0d1fefe4605b9d7f761e1e06f96; no voters:
I20260926 15:03:12.810113 15921 leader_election.cc:288] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:03:12.810223 15926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:03:12.810823 15926 raft_consensus.cc:686] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [term 1 LEADER]: Becoming Leader. State: Replica: c93ba0d1fefe4605b9d7f761e1e06f96, State: Running, Role: LEADER
I20260926 15:03:12.811321 15926 consensus_queue.cc:237] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } }
I20260926 15:03:12.812273 15921 sys_catalog.cc:564] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:03:12.813974 15927 sys_catalog.cc:455] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } }
I20260926 15:03:12.814153 15928 sys_catalog.cc:455] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c93ba0d1fefe4605b9d7f761e1e06f96. Latest consensus state: current_term: 1 leader_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } }
I20260926 15:03:12.814335 15927 sys_catalog.cc:458] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:12.814404 15928 sys_catalog.cc:458] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:12.816038 15933 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:03:12.821810 15933 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:03:12.830763 15933 catalog_manager.cc:1347] Generated new cluster ID: d728c8a2cf5643cc8a1ea2d38b1a354c
I20260926 15:03:12.830888 15933 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:03:12.853232 15933 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:03:12.854555 15933 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:03:12.863266 15933 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96: Generated new TSK 0
I20260926 15:03:12.864094 15933 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.123.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.123.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.123.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.65.keytab.
Entry for principal oryx/127.9.123.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.65.keytab.
I20260926 15:03:12.920877 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:32823
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.65.keytab
--principal=oryx/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:41353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:03:13.253872 15947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:13.254200 15947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:13.254343 15947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:13.261754 15947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:13.261888 15947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:13.262041 15947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:03:13.271919 15947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:41353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.65.keytab
--principal=oryx/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:32823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:13.273277 15947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:13.274590 15947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:13.287447 15955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:13.287525 15952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:13.287525 15953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:13.288151 15947 server_base.cc:1034] running on GCE node
I20260926 15:03:13.288775 15947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:13.289779 15947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:13.291007 15947 hybrid_clock.cc:648] HybridClock initialized: now 1790434993290943 us; error 58 us; skew 500 ppm
I20260926 15:03:13.291360 15947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:13 dist-test-slave-7153 krb5kdc[15841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434993, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:13.301172 15947 init.cc:377] Logged in from keytab as oryx/127.9.123.65@KRBTEST.COM (short username oryx)
I20260926 15:03:13.302861 15947 webserver.cc:460] Webserver started at http://127.9.123.65:38755/ using document root <none> and password file <none>
I20260926 15:03:13.303571 15947 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:13.303689 15947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:13.303974 15947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:13.307034 15947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "fedeebde7a6744c69356ec6652b19d3d"
format_stamp: "Formatted at 2026-09-26 15:03:13 on dist-test-slave-7153"
server_key: "82813239964d21a0b7ef730f7ffaa914"
server_key_iv: "04b0dcda9ca9f6c8d97e0f32e6e9eb32"
server_key_version: "encryptionkey@0"
I20260926 15:03:13.307852 15947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "fedeebde7a6744c69356ec6652b19d3d"
format_stamp: "Formatted at 2026-09-26 15:03:13 on dist-test-slave-7153"
server_key: "82813239964d21a0b7ef730f7ffaa914"
server_key_iv: "04b0dcda9ca9f6c8d97e0f32e6e9eb32"
server_key_version: "encryptionkey@0"
I20260926 15:03:13.313589 15947 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260926 15:03:13.317265 15962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:13.318753 15947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:03:13.318924 15947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "fedeebde7a6744c69356ec6652b19d3d"
format_stamp: "Formatted at 2026-09-26 15:03:13 on dist-test-slave-7153"
server_key: "82813239964d21a0b7ef730f7ffaa914"
server_key_iv: "04b0dcda9ca9f6c8d97e0f32e6e9eb32"
server_key_version: "encryptionkey@0"
I20260926 15:03:13.319124 15947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:13.338086 15947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:13.342109 15947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:13.342387 15947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:13.343326 15947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:13.344743 15947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:13.344830 15947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:13.344928 15947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:13.344980 15947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:13.374281 15947 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:46053
I20260926 15:03:13.374343 16074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:46053 every 8 connection(s)
I20260926 15:03:13.375672 15947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:03:13.383200 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 15947
I20260926 15:03:13.383406 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:03:13.384001 9709 external_mini_cluster.cc:1428] Setting key a8ab1813bc670b8a9dc5592555d0833e
Sep 26 15:03:13 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434993, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM
I20260926 15:03:13.395560 16077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:13.378150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:36457 (local address 127.9.123.126:32823)
0926 15:03:13.378614 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:13.378619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:13.379744 (+ 1125us) server_negotiation.cc:407] Connection header received
0926 15:03:13.381604 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:13.381610 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:13.381698 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:13.381794 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:13.383394 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:13.384175 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:13.385838 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:13.386238 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:13.386351 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:13.389945 (+ 3594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:13.389950 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:13.389954 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:13.389987 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:13.392490 (+ 2503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:13.393292 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:13.393296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:13.393298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:13.393422 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:13.393846 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:13.393850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:13.393852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:13.394115 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:13.394254 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:13.394984 (+ 730us) server_negotiation.cc:299] Negotiation successful
0926 15:03:13.395220 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":251,"threads_started":1}
I20260926 15:03:13.396615 16078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:13.378794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.9.123.65:36457)
0926 15:03:13.379544 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:13.379590 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:13.381299 (+ 1709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:13.382059 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:13.382068 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:13.382601 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:13.383224 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:13.383242 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:13.384322 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:13.384326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:13.385697 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:13.385710 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:13.386464 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:13.386472 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:13.386701 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:13.387347 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:13.387375 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:13.389704 (+ 2329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:13.392647 (+ 2943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:13.392655 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:13.392675 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:13.393163 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:13.393529 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:13.393534 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:13.393536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:13.393733 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:13.394245 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:13.394254 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:13.394606 (+ 352us) client_negotiation.cc:769] Sending connection context
0926 15:03:13.394859 (+ 253us) client_negotiation.cc:240] Negotiation successful
0926 15:03:13.395395 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":285,"threads_started":1}
I20260926 15:03:13.398317 16075 heartbeater.cc:347] Connected to a master server at 127.9.123.126:32823
I20260926 15:03:13.398619 16075 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:13.399374 16075 heartbeater.cc:511] Master 127.9.123.126:32823 requested a full tablet report, sending...
I20260926 15:03:13.401889 15886 ts_manager.cc:194] Registered new tserver with Master: fedeebde7a6744c69356ec6652b19d3d (127.9.123.65:46053)
WARNING: no policy specified for oryx/127.9.123.66@KRBTEST.COM; defaulting to no policy
I20260926 15:03:13.403678 15886 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.123.65@KRBTEST.COM'} at 127.9.123.65:36457
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.123.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.123.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.66.keytab.
Entry for principal oryx/127.9.123.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.66.keytab.
I20260926 15:03:13.429034 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:32823
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.66.keytab
--principal=oryx/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:41353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:03:13.754490 16081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:13.754789 16081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:13.754902 16081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:13.762374 16081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:13.762498 16081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:13.762651 16081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:03:13.772845 16081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:41353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.66.keytab
--principal=oryx/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:32823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:13.774127 16081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:13.775411 16081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:13.788800 16086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:13.788866 16087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:13.788841 16089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:13.789434 16081 server_base.cc:1034] running on GCE node
I20260926 15:03:13.790102 16081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:13.791115 16081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:13.792330 16081 hybrid_clock.cc:648] HybridClock initialized: now 1790434993792268 us; error 57 us; skew 500 ppm
I20260926 15:03:13.792645 16081 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:13 dist-test-slave-7153 krb5kdc[15841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434993, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:13.795779 16081 init.cc:377] Logged in from keytab as oryx/127.9.123.66@KRBTEST.COM (short username oryx)
I20260926 15:03:13.797423 16081 webserver.cc:460] Webserver started at http://127.9.123.66:41585/ using document root <none> and password file <none>
I20260926 15:03:13.798128 16081 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:13.798242 16081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:13.798511 16081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:13.801461 16081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "7b83c187309b40c8ae16f8149b7ad171"
format_stamp: "Formatted at 2026-09-26 15:03:13 on dist-test-slave-7153"
server_key: "e40d5c0f1b2a6ae6109ce84774014a7f"
server_key_iv: "7b1eb3e2759e42226be55feaf4e67d9c"
server_key_version: "encryptionkey@0"
I20260926 15:03:13.802214 16081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "7b83c187309b40c8ae16f8149b7ad171"
format_stamp: "Formatted at 2026-09-26 15:03:13 on dist-test-slave-7153"
server_key: "e40d5c0f1b2a6ae6109ce84774014a7f"
server_key_iv: "7b1eb3e2759e42226be55feaf4e67d9c"
server_key_version: "encryptionkey@0"
I20260926 15:03:13.807399 16081 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 15:03:13.811017 16096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:13.812469 16081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 15:03:13.812630 16081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "7b83c187309b40c8ae16f8149b7ad171"
format_stamp: "Formatted at 2026-09-26 15:03:13 on dist-test-slave-7153"
server_key: "e40d5c0f1b2a6ae6109ce84774014a7f"
server_key_iv: "7b1eb3e2759e42226be55feaf4e67d9c"
server_key_version: "encryptionkey@0"
I20260926 15:03:13.812819 16081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:13.830588 16081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:13.834729 16081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:13.835002 16081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:13.835958 16081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:13.837347 16081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:13.837428 16081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:13.837543 16081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:13.837594 16081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:13.867075 16081 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:43577
I20260926 15:03:13.867116 16208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:43577 every 8 connection(s)
I20260926 15:03:13.868347 16081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:03:13.878674 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 16081
I20260926 15:03:13.878860 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:03:13.879396 9709 external_mini_cluster.cc:1428] Setting key ce277625310040cc3ab6c26d5e2b6055
Sep 26 15:03:13 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434993, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM
I20260926 15:03:13.888145 16077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:13.870886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:56611 (local address 127.9.123.126:32823)
0926 15:03:13.871041 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:13.871045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:13.872307 (+ 1262us) server_negotiation.cc:407] Connection header received
0926 15:03:13.874106 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:13.874111 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:13.874187 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:13.874284 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:13.875796 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:13.876497 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:13.878186 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:13.878642 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:13.878756 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:13.882590 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:13.882596 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:13.882599 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:13.882634 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:13.885034 (+ 2400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:13.885827 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:13.885831 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:13.885834 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:13.885958 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:13.886388 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:13.886392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:13.886394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:13.886661 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:13.886808 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:13.887565 (+ 757us) server_negotiation.cc:299] Negotiation successful
0926 15:03:13.887803 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 15:03:13.889252 16211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:13.871415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.9.123.66:56611)
0926 15:03:13.872132 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:13.872172 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:13.873836 (+ 1664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:13.874480 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:13.874488 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:13.875030 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:13.875619 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:13.875647 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:13.876617 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:13.876621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:13.878010 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:13.878023 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:13.878913 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:13.878922 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:13.879237 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:13.879961 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:13.879997 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:13.882348 (+ 2351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:13.885208 (+ 2860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:13.885219 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:13.885241 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:13.885696 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:13.886065 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:13.886070 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:13.886074 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:13.886273 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:13.886786 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:13.886794 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:13.887194 (+ 400us) client_negotiation.cc:769] Sending connection context
0926 15:03:13.887448 (+ 254us) client_negotiation.cc:240] Negotiation successful
0926 15:03:13.888014 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":281,"threads_started":1}
I20260926 15:03:13.890945 16209 heartbeater.cc:347] Connected to a master server at 127.9.123.126:32823
I20260926 15:03:13.891268 16209 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:13.892040 16209 heartbeater.cc:511] Master 127.9.123.126:32823 requested a full tablet report, sending...
I20260926 15:03:13.893628 15886 ts_manager.cc:194] Registered new tserver with Master: 7b83c187309b40c8ae16f8149b7ad171 (127.9.123.66:43577)
I20260926 15:03:13.894477 15886 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.123.66@KRBTEST.COM'} at 127.9.123.66:56611
WARNING: no policy specified for oryx/127.9.123.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.123.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.123.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.67.keytab.
Entry for principal oryx/127.9.123.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.67.keytab.
I20260926 15:03:13.925365 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:32823
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.67.keytab
--principal=oryx/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:41353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:03:14.256857 16214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:14.257226 16214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:14.257404 16214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:14.264833 16214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:14.264961 16214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:14.265143 16214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:03:14.275065 16214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:41353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.67.keytab
--principal=oryx/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:32823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:14.276389 16214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:14.277804 16214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:14.291214 16220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:14.291234 16219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:14.291174 16222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:14.292116 16214 server_base.cc:1034] running on GCE node
I20260926 15:03:14.292753 16214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:14.293756 16214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:14.294960 16214 hybrid_clock.cc:648] HybridClock initialized: now 1790434994294905 us; error 44 us; skew 500 ppm
I20260926 15:03:14.295290 16214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:14 dist-test-slave-7153 krb5kdc[15841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434994, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:14.298509 16214 init.cc:377] Logged in from keytab as oryx/127.9.123.67@KRBTEST.COM (short username oryx)
I20260926 15:03:14.299990 16214 webserver.cc:460] Webserver started at http://127.9.123.67:46341/ using document root <none> and password file <none>
I20260926 15:03:14.300696 16214 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:14.300804 16214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:14.301084 16214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:14.304075 16214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "b0a766fb9dc74a42967c0463588f3257"
format_stamp: "Formatted at 2026-09-26 15:03:14 on dist-test-slave-7153"
server_key: "6ca2cbff094fd35d0c0f8982890b93f9"
server_key_iv: "caaac7fd17ec82f0b39aa0bffd539bc9"
server_key_version: "encryptionkey@0"
I20260926 15:03:14.305083 16214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "b0a766fb9dc74a42967c0463588f3257"
format_stamp: "Formatted at 2026-09-26 15:03:14 on dist-test-slave-7153"
server_key: "6ca2cbff094fd35d0c0f8982890b93f9"
server_key_iv: "caaac7fd17ec82f0b39aa0bffd539bc9"
server_key_version: "encryptionkey@0"
I20260926 15:03:14.311020 16214 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 15:03:14.314877 16229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:14.316465 16214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 15:03:14.316639 16214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "b0a766fb9dc74a42967c0463588f3257"
format_stamp: "Formatted at 2026-09-26 15:03:14 on dist-test-slave-7153"
server_key: "6ca2cbff094fd35d0c0f8982890b93f9"
server_key_iv: "caaac7fd17ec82f0b39aa0bffd539bc9"
server_key_version: "encryptionkey@0"
I20260926 15:03:14.316841 16214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:14.334125 16214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:14.338517 16214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:14.338795 16214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:14.339774 16214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:14.341173 16214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:14.341249 16214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:14.341375 16214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:14.341423 16214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:14.372195 16214 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:40451
I20260926 15:03:14.372246 16341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:40451 every 8 connection(s)
I20260926 15:03:14.373471 16214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:03:14.375319 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 16214
I20260926 15:03:14.375476 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:03:14.376045 9709 external_mini_cluster.cc:1428] Setting key 4688e1d52365f9772625a3a8a321b9d3
Sep 26 15:03:14 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434994, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM
I20260926 15:03:14.395047 16077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:14.376316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:42075 (local address 127.9.123.126:32823)
0926 15:03:14.376500 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:14.376506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:14.378035 (+ 1529us) server_negotiation.cc:407] Connection header received
0926 15:03:14.380405 (+ 2370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:14.380410 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:14.380499 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:14.380595 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:14.382312 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.383042 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.384796 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.385285 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.385390 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:14.389301 (+ 3911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:14.389308 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:14.389312 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:14.389357 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:14.392076 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:14.392897 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:14.392901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:14.392904 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:14.393042 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:14.393553 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:14.393557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:14.393560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:14.393838 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:14.393948 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:14.394626 (+ 678us) server_negotiation.cc:299] Negotiation successful
0926 15:03:14.394794 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 15:03:14.396075 16344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:14.376959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.9.123.67:42075)
0926 15:03:14.377868 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:14.377919 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:14.380143 (+ 2224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:14.380889 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:14.380898 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:14.381518 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:14.382142 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:14.382162 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.383282 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.383286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:14.384631 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:14.384645 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.385515 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.385521 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:14.385752 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:14.386496 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:14.386525 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:14.388978 (+ 2453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:14.392239 (+ 3261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:14.392248 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:14.392264 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:14.392745 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:14.393216 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:14.393220 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:14.393222 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:14.393425 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:14.393932 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:14.393938 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:14.394310 (+ 372us) client_negotiation.cc:769] Sending connection context
0926 15:03:14.394541 (+ 231us) client_negotiation.cc:240] Negotiation successful
0926 15:03:14.394941 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":325,"threads_started":1}
I20260926 15:03:14.397792 16342 heartbeater.cc:347] Connected to a master server at 127.9.123.126:32823
I20260926 15:03:14.398132 16342 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:14.398940 16342 heartbeater.cc:511] Master 127.9.123.126:32823 requested a full tablet report, sending...
I20260926 15:03:14.400544 15886 ts_manager.cc:194] Registered new tserver with Master: b0a766fb9dc74a42967c0463588f3257 (127.9.123.67:40451)
I20260926 15:03:14.401460 15886 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.123.67@KRBTEST.COM'} at 127.9.123.67:42075
I20260926 15:03:14.403960 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 15:03:14.407006 16075 heartbeater.cc:503] Master 127.9.123.126:32823 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.9.123.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.123.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.123.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.125.keytab.
Entry for principal oryx/127.9.123.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.125.keytab.
I20260926 15:03:14.449709 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.125:36901
--webserver_interface=127.9.123.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.125.keytab
--principal=oryx/127.9.123.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:41353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:32823,127.9.123.125:36901
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:03:14.769871 16347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:14.770275 16347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:14.770421 16347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:14.781929 16347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:03:14.782042 16347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:14.782096 16347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:14.782150 16347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:03:14.782192 16347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:03:14.791957 16347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:41353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:32823,127.9.123.125:36901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.125.keytab
--principal=oryx/127.9.123.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.125:36901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.123.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
W20260926 15:03:14.792284 16347 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.9.123.126:32823,127.9.123.125:36901'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260926 15:03:14.793291 16347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:14.794517 16347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:14.805619 16352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:14.805662 16353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:14.805677 16355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:14.806917 16347 server_base.cc:1034] running on GCE node
I20260926 15:03:14.807777 16347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:14.809283 16347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:14.810503 16347 hybrid_clock.cc:648] HybridClock initialized: now 1790434994810429 us; error 57 us; skew 500 ppm
I20260926 15:03:14.810830 16347 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:14 dist-test-slave-7153 krb5kdc[15841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434994, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:14.814132 16347 init.cc:377] Logged in from keytab as oryx/127.9.123.125@KRBTEST.COM (short username oryx)
I20260926 15:03:14.815797 16347 webserver.cc:460] Webserver started at http://127.9.123.125:39323/ using document root <none> and password file <none>
I20260926 15:03:14.816630 16347 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:14.816735 16347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:14.817041 16347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:14.820125 16347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/data/instance:
uuid: "3f2d4d6b2df742a280bec1cc4ef4f615"
format_stamp: "Formatted at 2026-09-26 15:03:14 on dist-test-slave-7153"
server_key: "2a7bac278bbe8f3b4a33dc2c342f7ede"
server_key_iv: "16c854c712bf8296af4b818d47ae2c9b"
server_key_version: "encryptionkey@0"
I20260926 15:03:14.820961 16347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/wal/instance:
uuid: "3f2d4d6b2df742a280bec1cc4ef4f615"
format_stamp: "Formatted at 2026-09-26 15:03:14 on dist-test-slave-7153"
server_key: "2a7bac278bbe8f3b4a33dc2c342f7ede"
server_key_iv: "16c854c712bf8296af4b818d47ae2c9b"
server_key_version: "encryptionkey@0"
I20260926 15:03:14.826502 16347 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 15:03:14.830178 16362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:14.831646 16347 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:03:14.831818 16347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/wal
uuid: "3f2d4d6b2df742a280bec1cc4ef4f615"
format_stamp: "Formatted at 2026-09-26 15:03:14 on dist-test-slave-7153"
server_key: "2a7bac278bbe8f3b4a33dc2c342f7ede"
server_key_iv: "16c854c712bf8296af4b818d47ae2c9b"
server_key_version: "encryptionkey@0"
I20260926 15:03:14.832038 16347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:14.849329 16347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:14.853559 16347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:14.853811 16347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:14.871065 16347 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.125:36901
I20260926 15:03:14.871088 16413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.125:36901 every 8 connection(s)
I20260926 15:03:14.872473 16347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/data/info.pb
I20260926 15:03:14.876281 16414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:14.878823 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 16347
I20260926 15:03:14.884327 16414 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:03:14 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.123.125@KRBTEST.COM
Sep 26 15:03:14 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434994, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.125@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM
I20260926 15:03:14.897116 16209 heartbeater.cc:503] Master 127.9.123.126:32823 was elected leader, sending a full tablet report...
I20260926 15:03:14.902441 16077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:14.887132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49250 (local address 127.9.123.126:32823)
0926 15:03:14.887333 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:14.887338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:14.887830 (+ 492us) server_negotiation.cc:407] Connection header received
0926 15:03:14.887987 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:14.887990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:14.888053 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:14.888137 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:14.889315 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.890127 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.891842 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.892503 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.892704 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:14.896410 (+ 3706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:14.896416 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:14.896419 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:14.896455 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:14.899656 (+ 3201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:14.900448 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:14.900454 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:14.900457 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:14.900593 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:14.900945 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:14.900950 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:14.900954 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:14.901315 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:14.901443 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:14.901907 (+ 464us) server_negotiation.cc:299] Negotiation successful
0926 15:03:14.902126 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 15:03:14.903389 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:14.880941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49494 (local address 127.9.123.125:36901)
0926 15:03:14.881800 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:14.881817 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:14.881869 (+ 52us) server_negotiation.cc:407] Connection header received
0926 15:03:14.884054 (+ 2185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:14.884085 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:14.884619 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:14.885156 (+ 537us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:14.886521 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.888313 (+ 1792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.889696 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.890468 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.890668 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:14.895165 (+ 4497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:14.895174 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:14.895188 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:14.895220 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:14.898671 (+ 3451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:14.899664 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:14.899672 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:14.899680 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:14.899918 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:14.900368 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:14.900372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:14.900374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:14.900984 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:14.901302 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:14.901713 (+ 411us) server_negotiation.cc:299] Negotiation successful
0926 15:03:14.902165 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"spinlock_wait_cycles":22400,"thread_start_us":352,"threads_started":1}
I20260926 15:03:14.903409 16418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:14.886988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.0.0.1:49250)
0926 15:03:14.887687 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:14.887730 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:14.887888 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:14.888224 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:14.888235 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:14.888899 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:14.889164 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:14.889175 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.890296 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.890301 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:14.891658 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:14.891670 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.892804 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.892809 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:14.892938 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:14.893664 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:14.893689 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:14.896100 (+ 2411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:14.899783 (+ 3683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:14.899796 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:14.899804 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:14.900349 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:14.900704 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:14.900709 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:14.900713 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:14.900847 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:14.901439 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:14.901446 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:14.901635 (+ 189us) client_negotiation.cc:769] Sending connection context
0926 15:03:14.901807 (+ 172us) client_negotiation.cc:240] Negotiation successful
0926 15:03:14.902313 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":132,"thread_start_us":272,"threads_started":1}
I20260926 15:03:14.906234 16414 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:03:14 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434994, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.125@KRBTEST.COM for oryx/127.9.123.125@KRBTEST.COM
I20260926 15:03:14.918071 16418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:14.906984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:36901 (local address 127.0.0.1:49498)
0926 15:03:14.907156 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:14.907174 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:14.907265 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:14.907491 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:14.907494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:14.907796 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:14.907928 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:14.907936 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.908815 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.908819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:14.909740 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:14.909752 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.910363 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.910367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:14.910524 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:14.911035 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:14.911052 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:14.913039 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:14.915750 (+ 2711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:14.915754 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:14.915757 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:14.916124 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:14.916478 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:14.916483 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:14.916486 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:14.916627 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:14.917240 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:14.917244 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:14.917398 (+ 154us) client_negotiation.cc:769] Sending connection context
0926 15:03:14.917527 (+ 129us) client_negotiation.cc:240] Negotiation successful
0926 15:03:14.917780 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 15:03:14.918227 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:14.907129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49498 (local address 127.9.123.125:36901)
0926 15:03:14.907258 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:14.907264 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:14.907279 (+ 15us) server_negotiation.cc:407] Connection header received
0926 15:03:14.907356 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:14.907359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:14.907410 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:14.907483 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:14.908029 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.908688 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.909865 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.910240 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.910334 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:14.913228 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:14.913232 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:14.913236 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:14.913269 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:14.915610 (+ 2341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:14.916227 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:14.916231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:14.916233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:14.916349 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:14.916782 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:14.916786 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:14.916788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:14.917136 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:14.917275 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:14.917601 (+ 326us) server_negotiation.cc:299] Negotiation successful
0926 15:03:14.917930 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":134,"server-negotiator.queue_time_us":43}
I20260926 15:03:14.923127 16414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615: Bootstrap starting.
I20260926 15:03:14.926513 16414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:14.927717 16414 log.cc:824] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:14.930563 16414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615: No bootstrap required, opened a new log
I20260926 15:03:14.937963 16414 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } }
I20260926 15:03:14.938313 16414 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:14.938407 16414 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f2d4d6b2df742a280bec1cc4ef4f615, State: Initialized, Role: FOLLOWER
I20260926 15:03:14.938966 16414 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } }
I20260926 15:03:14.939733 16420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } } }
I20260926 15:03:14.940021 16420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:03:14.940778 16414 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 15:03:14.949240 16431 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 15:03:14.949401 16431 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 15:03:14.951617 16347 master_runner.cc:428] Detected that this master 3f2d4d6b2df742a280bec1cc4ef4f615 is joining an existing cluster
I20260926 15:03:14.951730 16347 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.123.125:36901
I20260926 15:03:14.971758 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:14.956187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49504 (local address 127.9.123.125:36901)
0926 15:03:14.956373 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:14.956383 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:14.957435 (+ 1052us) server_negotiation.cc:407] Connection header received
0926 15:03:14.957641 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:14.957645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:14.957707 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:14.957773 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:14.959041 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.960043 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.962639 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.963251 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.963360 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:14.965680 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:14.965686 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:14.965690 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:14.965725 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:14.969067 (+ 3342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:14.969810 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:14.969815 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:14.969821 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:14.969951 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:14.970263 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:14.970267 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:14.970269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:14.970657 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:14.970796 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:14.971189 (+ 393us) server_negotiation.cc:299] Negotiation successful
0926 15:03:14.971398 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56,"spinlock_wait_cycles":40320}
I20260926 15:03:14.972391 16077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:14.955691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49252 (local address 127.9.123.126:32823)
0926 15:03:14.955877 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:14.955884 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:14.956519 (+ 635us) server_negotiation.cc:407] Connection header received
0926 15:03:14.956794 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:14.956800 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:14.956874 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:14.956952 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:14.958520 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.959631 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.962592 (+ 2961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.963232 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.963334 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:14.966017 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:14.966024 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:14.966028 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:14.966069 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:14.969255 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:14.970074 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:14.970080 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:14.970083 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:14.970246 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:14.970704 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:14.970709 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:14.970713 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:14.971043 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:14.971170 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:14.971817 (+ 647us) server_negotiation.cc:299] Negotiation successful
0926 15:03:14.972045 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 15:03:14.972760 16438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:14.956123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:36901 (local address 127.0.0.1:49504)
0926 15:03:14.957355 (+ 1232us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:14.957372 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:14.957522 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:14.957934 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:14.957938 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:14.958587 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:14.958896 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:14.958907 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.960257 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.960265 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:14.962402 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:14.962416 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.963603 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.963607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:14.963795 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:14.964390 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:14.964409 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:14.965509 (+ 1100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:14.969221 (+ 3712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:14.969225 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:14.969229 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:14.969697 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:14.970049 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:14.970053 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:14.970056 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:14.970181 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:14.970852 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:14.970856 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:14.970983 (+ 127us) client_negotiation.cc:769] Sending connection context
0926 15:03:14.972113 (+ 1130us) client_negotiation.cc:240] Negotiation successful
0926 15:03:14.972375 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1120,"mutex_wait_us":613,"thread_start_us":961,"threads_started":1}
I20260926 15:03:14.973740 16437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:14.955713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.0.0.1:49252)
0926 15:03:14.956414 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:14.956436 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:14.956628 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:14.957201 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:14.957206 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:14.957891 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:14.958340 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:14.958357 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.959817 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.959822 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:14.961554 (+ 1732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:14.961569 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.963644 (+ 2075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.963664 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:14.963856 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:14.964544 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:14.964566 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:14.965798 (+ 1232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:14.969415 (+ 3617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:14.969420 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:14.969424 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:14.969936 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:14.970386 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:14.970391 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:14.970395 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:14.970558 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:14.971253 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:14.971257 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:14.971465 (+ 208us) client_negotiation.cc:769] Sending connection context
0926 15:03:14.972961 (+ 1496us) client_negotiation.cc:240] Negotiation successful
0926 15:03:14.973308 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":1,"thread_start_us":655,"threads_started":1}
WARNING: no policy specified for oryx/127.9.123.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.123.124@KRBTEST.COM" created.
Sep 26 15:03:14 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434992, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.126@KRBTEST.COM for oryx/127.9.123.125@KRBTEST.COM
I20260926 15:03:14.989815 16420 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:03:14.990114 16420 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } }
I20260926 15:03:14.992238 16420 leader_election.cc:288] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c93ba0d1fefe4605b9d7f761e1e06f96 (127.9.123.126:32823)
I20260926 15:03:14.993064 15896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" is_pre_election: true
I20260926 15:03:14.994218 16365 leader_election.cc:302] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 3f2d4d6b2df742a280bec1cc4ef4f615; no voters: c93ba0d1fefe4605b9d7f761e1e06f96
I20260926 15:03:14.995045 16420 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 15:03:14.997016 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:14.980229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49520 (local address 127.9.123.125:36901)
0926 15:03:14.980391 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:14.980397 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:14.980602 (+ 205us) server_negotiation.cc:407] Connection header received
0926 15:03:14.980865 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:14.980873 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:14.980952 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:14.981027 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:14.982086 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.982909 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.984818 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.985563 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.985669 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:14.989868 (+ 4199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:14.989875 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:14.989880 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:14.989925 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:14.993548 (+ 3623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:14.994458 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:14.994464 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:14.994467 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:14.994810 (+ 343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:14.995262 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:14.995267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:14.995270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:14.995627 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:14.995757 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:14.996402 (+ 645us) server_negotiation.cc:299] Negotiation successful
0926 15:03:14.996663 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 15:03:14.997988 16440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:14.979881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:36901 (local address 127.0.0.1:49520)
0926 15:03:14.980494 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:14.980523 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:14.980692 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:14.981093 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:14.981128 (+ 35us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:14.981732 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:14.981942 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:14.981951 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.983048 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.983054 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:14.984643 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:14.984668 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:14.985929 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:14.985935 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:14.986143 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:14.986818 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:14.986842 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:14.989629 (+ 2787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:14.993724 (+ 4095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:14.993738 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:14.993746 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:14.994317 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:14.994951 (+ 634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:14.994956 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:14.994960 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:14.995134 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:14.995871 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:14.995880 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:14.996071 (+ 191us) client_negotiation.cc:769] Sending connection context
0926 15:03:14.997208 (+ 1137us) client_negotiation.cc:240] Negotiation successful
0926 15:03:14.997570 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":239,"threads_started":1}
I20260926 15:03:15.000202 15886 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.123.125:36901: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615"
member_type: NON_VOTER
last_known_addr {
host: "127.9.123.125"
port: 36901
}
attrs {
promote: true
}
}
dest_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96"
cas_config_opid_index: -1
I20260926 15:03:15.001503 15886 consensus_queue.cc:237] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: NON_VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: true } }
I20260926 15:03:15.004889 16444 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 3f2d4d6b2df742a280bec1cc4ef4f615 (127.9.123.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: NON_VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: true } } }
I20260926 15:03:15.005729 16389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 0 FOLLOWER]: Advancing to term 1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.123.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.124.keytab.
Entry for principal oryx/127.9.123.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.124.keytab.
I20260926 15:03:15.008615 16389 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 1 FOLLOWER]: Refusing update from remote peer c93ba0d1fefe4605b9d7f761e1e06f96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260926 15:03:15.009258 16444 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.123.125:36901
I20260926 15:03:15.009727 16444 sys_catalog.cc:455] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: NON_VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: true } } }
I20260926 15:03:15.009969 16444 sys_catalog.cc:458] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:15.010351 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.124:45251
--webserver_interface=127.9.123.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.124.keytab
--principal=oryx/127.9.123.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:41353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:32823,127.9.123.125:36901,127.9.123.124:45251
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:03:15.014482 16443 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: NON_VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:03:15.022022 16347 master.cc:537] Master@127.9.123.125:36901 shutting down...
W20260926 15:03:15.026110 15870 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260926 15:03:15.036727 16347 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:03:15.037344 16347 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:15.037611 16347 tablet_replica.cc:330] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615: stopping tablet replica
I20260926 15:03:15.044306 16347 master.cc:552] Master@127.9.123.125:36901 shutdown complete.
I20260926 15:03:15.044392 16347 master_runner.cc:305] Clearing existing system tablet
I20260926 15:03:15.044795 16347 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:15.044885 16347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:15.047058 16347 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260926 15:03:15.048408 16450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:15.048727 16347 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:15.049008 16347 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 15:03:15.049088 16347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/wal
uuid: "3f2d4d6b2df742a280bec1cc4ef4f615"
format_stamp: "Formatted at 2026-09-26 15:03:14 on dist-test-slave-7153"
server_key: "2a7bac278bbe8f3b4a33dc2c342f7ede"
server_key_iv: "16c854c712bf8296af4b818d47ae2c9b"
server_key_version: "encryptionkey@0"
I20260926 15:03:15.052587 16347 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260926 15:03:15.053555 15870 consensus_peers.cc:487] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 -> Peer 3f2d4d6b2df742a280bec1cc4ef4f615 (127.9.123.125:36901): Couldn't send request to peer 3f2d4d6b2df742a280bec1cc4ef4f615. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260926 15:03:15.057214 16347 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 15:03:15.057391 16347 log.cc:1196] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260926 15:03:15.057726 16347 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615: Deleting consensus metadata
I20260926 15:03:15.057941 16347 master_runner.cc:315] Copying system tablet from 127.9.123.126:32823
I20260926 15:03:15.062474 16347 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615: tablet copy: Beginning tablet copy session from remote peer at address 127.9.123.126:32823
I20260926 15:03:15.080000 16077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:15.063183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49262 (local address 127.9.123.126:32823)
0926 15:03:15.063389 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:15.063396 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:15.069688 (+ 6292us) server_negotiation.cc:407] Connection header received
0926 15:03:15.069880 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:15.069885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:15.069969 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:15.070053 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:15.070916 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.071622 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.072806 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.073275 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.073384 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:15.075078 (+ 1694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:15.075085 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:15.075089 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:15.075126 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:15.077621 (+ 2495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:15.078209 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:15.078214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:15.078218 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:15.078333 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:15.078676 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:15.078681 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:15.078684 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:15.078957 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:15.079082 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:15.079437 (+ 355us) server_negotiation.cc:299] Negotiation successful
0926 15:03:15.079642 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 15:03:15.080054 16456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:15.063128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.0.0.1:49262)
0926 15:03:15.069549 (+ 6421us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:15.069573 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:15.069749 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:15.070105 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:15.070109 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:15.070527 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:15.070764 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:15.070772 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.071759 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.071765 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:15.072656 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:15.072667 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.073425 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.073430 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:15.073607 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:15.074160 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:15.074176 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:15.074883 (+ 707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:15.077744 (+ 2861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:15.077753 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:15.077757 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:15.078099 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:15.078434 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:15.078439 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:15.078442 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:15.078582 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:15.079063 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:15.079068 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:15.079215 (+ 147us) client_negotiation.cc:769] Sending connection context
0926 15:03:15.079337 (+ 122us) client_negotiation.cc:240] Negotiation successful
0926 15:03:15.079642 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6269,"thread_start_us":6161,"threads_started":1}
I20260926 15:03:15.081312 15906 tablet_copy_service.cc:140] P c93ba0d1fefe4605b9d7f761e1e06f96: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 3f2d4d6b2df742a280bec1cc4ef4f615 ({username='oryx', principal='oryx/127.9.123.125@KRBTEST.COM'} at 127.0.0.1:49262)
I20260926 15:03:15.081538 15906 tablet_copy_service.cc:161] P c93ba0d1fefe4605b9d7f761e1e06f96: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 3f2d4d6b2df742a280bec1cc4ef4f615 at {username='oryx', principal='oryx/127.9.123.125@KRBTEST.COM'} at 127.0.0.1:49262: session id = 3f2d4d6b2df742a280bec1cc4ef4f615-00000000000000000000000000000000
I20260926 15:03:15.084455 15906 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 15:03:15.086555 16347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 15:03:15.091730 16347 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615: tablet copy: Starting download of 0 data blocks...
I20260926 15:03:15.092074 16347 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615: tablet copy: Starting download of 1 WAL segments...
I20260926 15:03:15.094693 16347 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 15:03:15.098125 15906 tablet_copy_service.cc:342] P c93ba0d1fefe4605b9d7f761e1e06f96: Request end of tablet copy session 3f2d4d6b2df742a280bec1cc4ef4f615-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.123.125@KRBTEST.COM'} at 127.0.0.1:49262
I20260926 15:03:15.098297 15906 tablet_copy_service.cc:433] P c93ba0d1fefe4605b9d7f761e1e06f96: ending tablet copy session 3f2d4d6b2df742a280bec1cc4ef4f615-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 3f2d4d6b2df742a280bec1cc4ef4f615
I20260926 15:03:15.107991 16347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:15.110443 16457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:15.120321 16458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:15.120945 16460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:15.121683 16347 server_base.cc:1034] running on GCE node
I20260926 15:03:15.122169 16347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:15.122812 16347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:15.124075 16347 hybrid_clock.cc:648] HybridClock initialized: now 1790434995124044 us; error 63 us; skew 500 ppm
I20260926 15:03:15.124289 16347 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:15 dist-test-slave-7153 krb5kdc[15841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434995, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:15.126094 16347 init.cc:377] Logged in from keytab as oryx/127.9.123.125@KRBTEST.COM (short username oryx)
I20260926 15:03:15.127259 16347 webserver.cc:460] Webserver started at http://127.9.123.125:35463/ using document root <none> and password file <none>
I20260926 15:03:15.127542 16347 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:15.127611 16347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:15.130000 16347 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260926 15:03:15.131457 16467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:15.131801 16347 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 15:03:15.131906 16347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/wal
uuid: "3f2d4d6b2df742a280bec1cc4ef4f615"
format_stamp: "Formatted at 2026-09-26 15:03:14 on dist-test-slave-7153"
server_key: "2a7bac278bbe8f3b4a33dc2c342f7ede"
server_key_iv: "16c854c712bf8296af4b818d47ae2c9b"
server_key_version: "encryptionkey@0"
I20260926 15:03:15.132093 16347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:15.149714 16347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:15.152966 16347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:15.186383 16347 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.125:36901
I20260926 15:03:15.186383 16518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.125:36901 every 8 connection(s)
I20260926 15:03:15.187332 16347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-1/data/info.pb
I20260926 15:03:15.194278 16519 sys_catalog.cc:263] Verifying existing consensus state
I20260926 15:03:15.196741 16519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615: Bootstrap starting.
I20260926 15:03:15.209626 16519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 15:03:15.210075 16519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615: Bootstrap complete.
I20260926 15:03:15.210892 16519 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: NON_VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: true } }
I20260926 15:03:15.211094 16519 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3f2d4d6b2df742a280bec1cc4ef4f615, State: Initialized, Role: LEARNER
I20260926 15:03:15.211305 16519 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: NON_VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: true } }
I20260926 15:03:15.212028 16522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: NON_VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: true } } }
I20260926 15:03:15.212288 16519 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:03:15.212558 16522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [sys.catalog]: This master's current role is: LEARNER
I20260926 15:03:15.219978 16533 catalog_manager.cc:1259] Loaded cluster ID: d728c8a2cf5643cc8a1ea2d38b1a354c
I20260926 15:03:15.220077 16533 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615: loading cluster ID for follower catalog manager: success
I20260926 15:03:15.222786 16533 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615: acquiring CA information for follower catalog manager: success
I20260926 15:03:15.224885 16533 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 15:03:15.404460 16342 heartbeater.cc:503] Master 127.9.123.126:32823 was elected leader, sending a full tablet report...
W20260926 15:03:15.434124 16447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:15.434432 16447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:15.434552 16447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:15.441860 16447 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:03:15.441988 16447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:15.442065 16447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:15.442134 16447 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:03:15.442191 16447 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:03:15.452526 16447 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:41353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:32823,127.9.123.125:36901,127.9.123.124:45251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.124.keytab
--principal=oryx/127.9.123.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.124:45251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.123.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:15.453902 16447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:15.455186 16447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:15.467412 16539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:15.467388 16541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:15.467952 16538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:15.468142 16447 server_base.cc:1034] running on GCE node
I20260926 15:03:15.468992 16447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:15.470497 16447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:15.471728 16447 hybrid_clock.cc:648] HybridClock initialized: now 1790434995471669 us; error 50 us; skew 500 ppm
I20260926 15:03:15.472096 16447 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:15 dist-test-slave-7153 krb5kdc[15841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434995, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:15.475144 16447 init.cc:377] Logged in from keytab as oryx/127.9.123.124@KRBTEST.COM (short username oryx)
I20260926 15:03:15.476524 16447 webserver.cc:460] Webserver started at http://127.9.123.124:43943/ using document root <none> and password file <none>
I20260926 15:03:15.477244 16447 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:15.477370 16447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:15.477660 16447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:15.480585 16447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/data/instance:
uuid: "b9febb6a22304d1f9376e29539155b4b"
format_stamp: "Formatted at 2026-09-26 15:03:15 on dist-test-slave-7153"
server_key: "83fc37e03f7ff376470e002d1bb7324d"
server_key_iv: "7fde51f017b4894ce0e96c9e8b438276"
server_key_version: "encryptionkey@0"
I20260926 15:03:15.481379 16447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/wal/instance:
uuid: "b9febb6a22304d1f9376e29539155b4b"
format_stamp: "Formatted at 2026-09-26 15:03:15 on dist-test-slave-7153"
server_key: "83fc37e03f7ff376470e002d1bb7324d"
server_key_iv: "7fde51f017b4894ce0e96c9e8b438276"
server_key_version: "encryptionkey@0"
I20260926 15:03:15.486860 16447 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260926 15:03:15.490526 16548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:15.491984 16447 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:03:15.492174 16447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/wal
uuid: "b9febb6a22304d1f9376e29539155b4b"
format_stamp: "Formatted at 2026-09-26 15:03:15 on dist-test-slave-7153"
server_key: "83fc37e03f7ff376470e002d1bb7324d"
server_key_iv: "7fde51f017b4894ce0e96c9e8b438276"
server_key_version: "encryptionkey@0"
I20260926 15:03:15.492465 16447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:15.526851 16447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:15.533270 16447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:15.533561 16447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:15.551992 16447 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.124:45251
I20260926 15:03:15.552014 16599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.124:45251 every 8 connection(s)
I20260926 15:03:15.553464 16447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/data/info.pb
I20260926 15:03:15.557315 16600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:15.563817 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 16447
I20260926 15:03:15.565392 16600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:03:15 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434995, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.124@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM
Sep 26 15:03:15 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.123.124@KRBTEST.COM
I20260926 15:03:15.584717 16077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:15.569208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49264 (local address 127.9.123.126:32823)
0926 15:03:15.569363 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:15.569367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:15.570095 (+ 728us) server_negotiation.cc:407] Connection header received
0926 15:03:15.570495 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:15.570501 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:15.570575 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:15.570661 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:15.571999 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.572761 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.574380 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.574779 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.574888 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:15.579010 (+ 4122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:15.579015 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:15.579019 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:15.579054 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:15.581601 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:15.582291 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:15.582295 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:15.582297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:15.582424 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:15.582853 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:15.582857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:15.582859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:15.583126 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:15.583249 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:15.584118 (+ 869us) server_negotiation.cc:299] Negotiation successful
0926 15:03:15.584368 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 15:03:15.585827 16603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:15.566483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47194 (local address 127.9.123.124:45251)
0926 15:03:15.567306 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:15.567320 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:15.567365 (+ 45us) server_negotiation.cc:407] Connection header received
0926 15:03:15.569728 (+ 2363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:15.569764 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:15.570262 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:15.570522 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:15.572081 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.573326 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.574612 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.575182 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.575354 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:15.579412 (+ 4058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:15.579420 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:15.579434 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:15.579467 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:15.581889 (+ 2422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:15.582515 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:15.582522 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:15.582524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:15.582732 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:15.583080 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:15.583084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:15.583086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:15.583670 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:15.583998 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:15.584338 (+ 340us) server_negotiation.cc:299] Negotiation successful
0926 15:03:15.584724 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":193,"server-negotiator.queue_time_us":591,"thread_start_us":341,"threads_started":1}
I20260926 15:03:15.585861 16604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:15.569133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.0.0.1:49264)
0926 15:03:15.569943 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:15.569980 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:15.570254 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:15.570745 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:15.570752 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:15.571419 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:15.571815 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:15.571838 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.572911 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.572915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:15.574220 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:15.574234 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.575009 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.575016 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:15.575222 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:15.575948 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:15.575971 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:15.578803 (+ 2832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:15.581730 (+ 2927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:15.581742 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:15.581756 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:15.582170 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:15.582522 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:15.582528 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:15.582532 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:15.582747 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:15.583248 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:15.583257 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:15.583690 (+ 433us) client_negotiation.cc:769] Sending connection context
0926 15:03:15.583995 (+ 305us) client_negotiation.cc:240] Negotiation successful
0926 15:03:15.584592 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"mutex_wait_us":3,"spinlock_wait_cycles":22272,"thread_start_us":427,"threads_started":1}
I20260926 15:03:15.587549 16600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:03:15 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434995, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.124@KRBTEST.COM for oryx/127.9.123.125@KRBTEST.COM
I20260926 15:03:15.601832 16607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:15.595122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:36901 (local address 127.0.0.1:49540)
0926 15:03:15.595622 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:15.595647 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:15.595763 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:15.596125 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:15.596130 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:15.596180 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:15.596348 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:15.596358 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.597876 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.597883 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:15.599365 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:15.599382 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.600856 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.600864 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:15.601061 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:15.601237 (+ 176us) client_negotiation.cc:769] Sending connection context
0926 15:03:15.601336 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 15:03:15.601486 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":217,"threads_started":1}
I20260926 15:03:15.601881 16608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:15.595280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49540 (local address 127.9.123.125:36901)
0926 15:03:15.595841 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:15.595846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:15.595862 (+ 16us) server_negotiation.cc:407] Connection header received
0926 15:03:15.595912 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:15.595916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:15.595977 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:15.596073 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:15.596499 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.597720 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.599513 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.600622 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.600768 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:15.600991 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:15.601367 (+ 376us) server_negotiation.cc:299] Negotiation successful
0926 15:03:15.601481 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":324,"threads_started":1}
I20260926 15:03:15.601899 16605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:15.588345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49526 (local address 127.9.123.125:36901)
0926 15:03:15.588915 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:15.588919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:15.588934 (+ 15us) server_negotiation.cc:407] Connection header received
0926 15:03:15.588978 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:15.588982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:15.589037 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:15.589170 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:15.589788 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.591181 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.592396 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.592808 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.592899 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:15.596152 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:15.596161 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:15.596166 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:15.596202 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:15.599054 (+ 2852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:15.599783 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:15.599788 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:15.599791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:15.599966 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:15.600346 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:15.600351 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:15.600355 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:15.600659 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:15.600799 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:15.601255 (+ 456us) server_negotiation.cc:299] Negotiation successful
0926 15:03:15.601463 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":338,"threads_started":1}
I20260926 15:03:15.602216 16604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:15.588195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:36901 (local address 127.0.0.1:49526)
0926 15:03:15.588364 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:15.588383 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:15.588483 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:15.589224 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:15.589227 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:15.589522 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:15.589666 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:15.589674 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.591325 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.591329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:15.592261 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:15.592273 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.592927 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.592932 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:15.593073 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:15.593658 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:15.593680 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:15.595955 (+ 2275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:15.599183 (+ 3228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:15.599188 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:15.599192 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:15.599693 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:15.600095 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:15.600099 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:15.600102 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:15.600239 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:15.600790 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:15.600795 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:15.600963 (+ 168us) client_negotiation.cc:769] Sending connection context
0926 15:03:15.601168 (+ 205us) client_negotiation.cc:240] Negotiation successful
0926 15:03:15.601883 (+ 715us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260926 15:03:15.603267 16494 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260926 15:03:15.603801 16600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.124" port: 45251 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:03:15.603844 16522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c93ba0d1fefe4605b9d7f761e1e06f96. Latest consensus state: current_term: 1 leader_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: NON_VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: true } } }
I20260926 15:03:15.604087 16522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [sys.catalog]: This master's current role is: LEARNER
Sep 26 15:03:15 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434995, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.124@KRBTEST.COM for oryx/127.9.123.124@KRBTEST.COM
I20260926 15:03:15.615615 16604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:15.604596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:45251 (local address 127.0.0.1:47206)
0926 15:03:15.604775 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:15.604794 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:15.604893 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:15.605203 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:15.605208 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:15.605536 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:15.605697 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:15.605705 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.606580 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.606584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:15.607404 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:15.607414 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.608026 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.608030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:15.608185 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:15.608725 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:15.608742 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:15.610702 (+ 1960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:15.613543 (+ 2841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:15.613547 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:15.613549 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:15.613914 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:15.614236 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:15.614239 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:15.614241 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:15.614383 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:15.614838 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:15.614843 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:15.614959 (+ 116us) client_negotiation.cc:769] Sending connection context
0926 15:03:15.615108 (+ 149us) client_negotiation.cc:240] Negotiation successful
0926 15:03:15.615324 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 15:03:15.615813 16603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:15.604815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47206 (local address 127.9.123.124:45251)
0926 15:03:15.604955 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:15.604959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:15.604974 (+ 15us) server_negotiation.cc:407] Connection header received
0926 15:03:15.605012 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:15.605015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:15.605068 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:15.605176 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:15.605805 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.606455 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.607520 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.607908 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.607999 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:15.610867 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:15.610871 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:15.610874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:15.610902 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:15.613402 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:15.614008 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:15.614011 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:15.614014 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:15.614139 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:15.614464 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:15.614467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:15.614469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:15.614727 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:15.614835 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:15.615177 (+ 342us) server_negotiation.cc:299] Negotiation successful
0926 15:03:15.615497 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":53}
I20260926 15:03:15.620551 16600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b: Bootstrap starting.
I20260926 15:03:15.624258 16600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:15.625281 16600 log.cc:824] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:15.627888 16600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b: No bootstrap required, opened a new log
I20260926 15:03:15.635164 16600 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 45251 } }
I20260926 15:03:15.635533 16600 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:15.635643 16600 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9febb6a22304d1f9376e29539155b4b, State: Initialized, Role: FOLLOWER
I20260926 15:03:15.636269 16600 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 45251 } }
I20260926 15:03:15.637208 16610 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 45251 } } }
I20260926 15:03:15.637501 16610 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:03:15.638250 16600 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:03:15.641402 16610 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:03:15.641661 16610 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 45251 } }
I20260926 15:03:15.643088 16610 leader_election.cc:288] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c93ba0d1fefe4605b9d7f761e1e06f96 (127.9.123.126:32823), 3f2d4d6b2df742a280bec1cc4ef4f615 (127.9.123.125:36901)
I20260926 15:03:15.643759 15896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b9febb6a22304d1f9376e29539155b4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" is_pre_election: true
I20260926 15:03:15.644284 16494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b9febb6a22304d1f9376e29539155b4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" is_pre_election: true
I20260926 15:03:15.644989 16549 leader_election.cc:302] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b9febb6a22304d1f9376e29539155b4b; no voters: 3f2d4d6b2df742a280bec1cc4ef4f615, c93ba0d1fefe4605b9d7f761e1e06f96
I20260926 15:03:15.645653 16610 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260926 15:03:15.648478 16621 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 15:03:15.648607 16621 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 15:03:15.651446 16447 master_runner.cc:428] Detected that this master b9febb6a22304d1f9376e29539155b4b is joining an existing cluster
I20260926 15:03:15.651548 16447 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.123.124:45251
I20260926 15:03:15.674863 16077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:15.655868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49268 (local address 127.9.123.126:32823)
0926 15:03:15.656026 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:15.656031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:15.657294 (+ 1263us) server_negotiation.cc:407] Connection header received
0926 15:03:15.657571 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:15.657578 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:15.657662 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:15.657735 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:15.659450 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.660565 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.664632 (+ 4067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.665359 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.665462 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:15.668347 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:15.668354 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:15.668359 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:15.668401 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:15.671636 (+ 3235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:15.672477 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:15.672483 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:15.672487 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:15.672665 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:15.673164 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:15.673169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:15.673172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:15.673520 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:15.673640 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:15.674272 (+ 632us) server_negotiation.cc:299] Negotiation successful
0926 15:03:15.674498 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260926 15:03:15.675789 16605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:15.655681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49544 (local address 127.9.123.125:36901)
0926 15:03:15.655862 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:15.655868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:15.660685 (+ 4817us) server_negotiation.cc:407] Connection header received
0926 15:03:15.660855 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:15.660860 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:15.660973 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:15.661058 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:15.662290 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.663315 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.665771 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.666419 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.666511 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:15.669338 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:15.669344 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:15.669348 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:15.669389 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:15.672936 (+ 3547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:15.673709 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:15.673713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:15.673716 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:15.673861 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:15.674270 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:15.674274 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:15.674277 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:15.674592 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:15.674726 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:15.675269 (+ 543us) server_negotiation.cc:299] Negotiation successful
0926 15:03:15.675477 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 15:03:15.676016 16603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:15.655729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47210 (local address 127.9.123.124:45251)
0926 15:03:15.655882 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:15.655885 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:15.660969 (+ 5084us) server_negotiation.cc:407] Connection header received
0926 15:03:15.661160 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:15.661165 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:15.661230 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:15.661305 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:15.662777 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.663938 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.666203 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.667003 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.667135 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:15.669485 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:15.669492 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:15.669496 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:15.669527 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:15.673205 (+ 3678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:15.673958 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:15.673963 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:15.673967 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:15.674105 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:15.674465 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:15.674470 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:15.674474 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:15.674800 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:15.674942 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:15.675412 (+ 470us) server_negotiation.cc:299] Negotiation successful
0926 15:03:15.675645 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":55}
I20260926 15:03:15.676671 16627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:15.655342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.0.0.1:49268)
0926 15:03:15.657173 (+ 1831us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:15.657196 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:15.657418 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:15.657955 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:15.657960 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:15.658797 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:15.659254 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:15.659269 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.662212 (+ 2943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.662216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:15.664456 (+ 2240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:15.664471 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.665733 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.665739 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:15.665943 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:15.666694 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:15.666721 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:15.668037 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:15.671796 (+ 3759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:15.671801 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:15.671805 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:15.672322 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:15.672820 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:15.672824 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:15.672828 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:15.673007 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:15.673726 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:15.673730 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:15.673984 (+ 254us) client_negotiation.cc:769] Sending connection context
0926 15:03:15.675424 (+ 1440us) client_negotiation.cc:240] Negotiation successful
0926 15:03:15.675824 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1644,"mutex_wait_us":70,"thread_start_us":1484,"threads_started":1}
I20260926 15:03:15.676776 16628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:15.655687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:36901 (local address 127.0.0.1:49544)
0926 15:03:15.660567 (+ 4880us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:15.660585 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:15.660719 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:15.661078 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:15.661082 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:15.661791 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:15.662122 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:15.662133 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.663489 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.663495 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:15.665598 (+ 2103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:15.665612 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.666837 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.666844 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:15.667062 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:15.667891 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:15.667914 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:15.669138 (+ 1224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:15.673067 (+ 3929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:15.673071 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:15.673075 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:15.673569 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:15.673996 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:15.674000 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:15.674003 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:15.674163 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:15.674799 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:15.674803 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:15.675002 (+ 199us) client_negotiation.cc:769] Sending connection context
0926 15:03:15.676019 (+ 1017us) client_negotiation.cc:240] Negotiation successful
0926 15:03:15.676334 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4752,"thread_start_us":4556,"threads_started":1}
I20260926 15:03:15.677122 16629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:15.655744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:45251 (local address 127.0.0.1:47210)
0926 15:03:15.660887 (+ 5143us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:15.660907 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:15.661038 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:15.661503 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:15.661507 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:15.662199 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:15.662638 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:15.662649 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.664522 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.664531 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:15.666055 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:15.666067 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.667377 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.667389 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:15.667551 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:15.668062 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:15.668085 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:15.669324 (+ 1239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:15.673339 (+ 4015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:15.673344 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:15.673348 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:15.673832 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:15.674224 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:15.674229 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:15.674232 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:15.674368 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:15.675003 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:15.675008 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:15.675162 (+ 154us) client_negotiation.cc:769] Sending connection context
0926 15:03:15.676241 (+ 1079us) client_negotiation.cc:240] Negotiation successful
0926 15:03:15.676519 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5019,"mutex_wait_us":45,"thread_start_us":5080,"threads_started":1}
Sep 26 15:03:15 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434992, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.126@KRBTEST.COM for oryx/127.9.123.124@KRBTEST.COM
I20260926 15:03:15.691830 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 15947
I20260926 15:03:15.696355 16607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:15.682589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:45251 (local address 127.0.0.1:47224)
0926 15:03:15.682754 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:15.682772 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:15.682871 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:15.683261 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:15.683267 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:15.683666 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:15.683804 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:15.683812 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.684784 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.684790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:15.685709 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:15.685721 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.686606 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.686616 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:15.686766 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:15.687317 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:15.687334 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:15.689642 (+ 2308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:15.693322 (+ 3680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:15.693328 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:15.693332 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:15.693859 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:15.694360 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:15.694365 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:15.694368 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:15.694536 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:15.695360 (+ 824us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:15.695365 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:15.695537 (+ 172us) client_negotiation.cc:769] Sending connection context
0926 15:03:15.695678 (+ 141us) client_negotiation.cc:240] Negotiation successful
0926 15:03:15.695984 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260926 15:03:15.696380 16603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:15.682626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47224 (local address 127.9.123.124:45251)
0926 15:03:15.682860 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:15.682865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:15.682884 (+ 19us) server_negotiation.cc:407] Connection header received
0926 15:03:15.682977 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:15.682981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:15.683106 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:15.683227 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:15.683940 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.684665 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.685840 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.686461 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.686577 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:15.689856 (+ 3279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:15.689862 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:15.689865 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:15.689897 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:15.693141 (+ 3244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:15.694035 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:15.694040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:15.694044 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:15.694213 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:15.694650 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:15.694653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:15.694655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:15.695233 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:15.695372 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:15.695800 (+ 428us) server_negotiation.cc:299] Negotiation successful
0926 15:03:15.696020 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 15:03:15.698154 15886 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.123.124:45251: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b9febb6a22304d1f9376e29539155b4b"
member_type: NON_VOTER
last_known_addr {
host: "127.9.123.124"
port: 45251
}
attrs {
promote: true
}
}
dest_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96"
cas_config_opid_index: 5
I20260926 15:03:15.699195 15886 consensus_queue.cc:237] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: NON_VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: true } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: NON_VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: true } }
I20260926 15:03:15.701515 16494 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 1 LEARNER]: Refusing update from remote peer c93ba0d1fefe4605b9d7f761e1e06f96: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 15:03:15.701821 16575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:15.700990 16606 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b9febb6a22304d1f9376e29539155b4b (127.9.123.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: NON_VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: true } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: NON_VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: true } } }
I20260926 15:03:15.705526 16631 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: NON_VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260926 15:03:15.708227 16494 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b9febb6a22304d1f9376e29539155b4b (127.9.123.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: NON_VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: true } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: NON_VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: true } } }
I20260926 15:03:15.709304 16575 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [term 1 FOLLOWER]: Refusing update from remote peer c93ba0d1fefe4605b9d7f761e1e06f96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260926 15:03:15.712837 16522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: NON_VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: true } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: NON_VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: true } } }
I20260926 15:03:15.713176 16522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [sys.catalog]: This master's current role is: LEARNER
Sep 26 15:03:15 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.123.65@KRBTEST.COM
I20260926 15:03:15.715003 16606 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.123.124:45251
I20260926 15:03:15.715742 16447 master.cc:537] Master@127.9.123.124:45251 shutting down...
I20260926 15:03:15.715595 16631 sys_catalog.cc:455] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: NON_VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: true } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: NON_VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: true } } }
I20260926 15:03:15.715905 16631 sys_catalog.cc:458] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:15.713925 16632 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: NON_VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.005s
I20260926 15:03:15.719269 16631 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96: attempting to promote NON_VOTER 3f2d4d6b2df742a280bec1cc4ef4f615 to VOTER
I20260926 15:03:15.721851 16631 consensus_queue.cc:237] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: NON_VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: true } }
I20260926 15:03:15.724483 16494 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 1 LEARNER]: Refusing update from remote peer c93ba0d1fefe4605b9d7f761e1e06f96: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260926 15:03:15.725230 16606 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260926 15:03:15.726094 15871 consensus_peers.cc:487] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 -> Peer b9febb6a22304d1f9376e29539155b4b (127.9.123.124:45251): Couldn't send request to peer b9febb6a22304d1f9376e29539155b4b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260926 15:03:15.726999 16630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:15.692872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44120 (local address 127.9.123.65:46053)
0926 15:03:15.693568 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:15.693578 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:15.693608 (+ 30us) server_negotiation.cc:407] Connection header received
0926 15:03:15.693678 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:15.693687 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:15.693990 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:15.694186 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:15.696366 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.698126 (+ 1760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.699899 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.708695 (+ 8796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.708882 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:15.714858 (+ 5976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:15.714877 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:15.714898 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:15.714944 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:15.720164 (+ 5220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:15.721260 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:15.721270 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:15.721274 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:15.721433 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:15.721937 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:15.721942 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:15.721945 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:15.722389 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:15.725962 (+ 3573us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:15.726157 (+ 195us) server_negotiation.cc:299] Negotiation successful
0926 15:03:15.726606 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":593,"threads_started":1}
I20260926 15:03:15.727864 16632 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 3f2d4d6b2df742a280bec1cc4ef4f615 (127.9.123.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: NON_VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: true } } }
I20260926 15:03:15.728721 16070 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:15.728540 16494 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 3f2d4d6b2df742a280bec1cc4ef4f615 (127.9.123.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: NON_VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: true } } }
I20260926 15:03:15.731832 16606 sys_catalog.cc:455] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: NON_VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: true } } }
I20260926 15:03:15.732131 16606 sys_catalog.cc:458] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:15.731855 16631 sys_catalog.cc:455] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: NON_VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: true } } }
I20260926 15:03:15.732712 16631 sys_catalog.cc:458] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:15.733448 16447 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:03:15.733879 16447 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:15.733879 16522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: NON_VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: true } } }
I20260926 15:03:15.734109 16447 tablet_replica.cc:330] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b: stopping tablet replica
I20260926 15:03:15.734119 16522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:03:15.741629 16447 master.cc:552] Master@127.9.123.124:45251 shutdown complete.
I20260926 15:03:15.741729 16447 master_runner.cc:305] Clearing existing system tablet
I20260926 15:03:15.742132 16447 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:15.742235 16447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:15.744427 16447 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.001s
I20260926 15:03:15.745865 16639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:15.746173 16447 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:15.746421 16447 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260926 15:03:15.746518 16447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/wal
uuid: "b9febb6a22304d1f9376e29539155b4b"
format_stamp: "Formatted at 2026-09-26 15:03:15 on dist-test-slave-7153"
server_key: "83fc37e03f7ff376470e002d1bb7324d"
server_key_iv: "7fde51f017b4894ce0e96c9e8b438276"
server_key_version: "encryptionkey@0"
I20260926 15:03:15.750243 16447 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:03:15.758369 16447 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260926 15:03:15.758642 16447 log.cc:1196] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260926 15:03:15.759332 16447 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b: Deleting consensus metadata
I20260926 15:03:15.759586 16447 master_runner.cc:315] Copying system tablet from 127.9.123.126:32823
I20260926 15:03:15.762213 16447 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b: tablet copy: Beginning tablet copy session from remote peer at address 127.9.123.126:32823
I20260926 15:03:15.780475 16077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:15.763037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49282 (local address 127.9.123.126:32823)
0926 15:03:15.763274 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:15.763280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:15.765701 (+ 2421us) server_negotiation.cc:407] Connection header received
0926 15:03:15.765887 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:15.765894 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:15.765991 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:15.766098 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:15.767251 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.768346 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.770305 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.770986 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.771088 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:15.773661 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:15.773668 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:15.773673 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:15.773713 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:15.777293 (+ 3580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:15.778136 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:15.778142 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:15.778147 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:15.778298 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:15.778747 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:15.778752 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:15.778756 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:15.779103 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:15.779251 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:15.779809 (+ 558us) server_negotiation.cc:299] Negotiation successful
0926 15:03:15.780067 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260926 15:03:15.781584 16645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:15.762971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.0.0.1:49282)
0926 15:03:15.765559 (+ 2588us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:15.765584 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:15.765755 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:15.766156 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:15.766161 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:15.766806 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:15.767085 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:15.767099 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.768509 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.768515 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:15.770133 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:15.770153 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:15.771379 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:15.771385 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:15.771599 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:15.772285 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:15.772308 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:15.773438 (+ 1130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:15.777482 (+ 4044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:15.777488 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:15.777492 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:15.777990 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:15.778435 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:15.778440 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:15.778444 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:15.778628 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:15.779333 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:15.779338 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:15.779513 (+ 175us) client_negotiation.cc:769] Sending connection context
0926 15:03:15.780754 (+ 1241us) client_negotiation.cc:240] Negotiation successful
0926 15:03:15.781144 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2448,"thread_start_us":2317,"threads_started":1}
I20260926 15:03:15.782150 15906 tablet_copy_service.cc:140] P c93ba0d1fefe4605b9d7f761e1e06f96: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b9febb6a22304d1f9376e29539155b4b ({username='oryx', principal='oryx/127.9.123.124@KRBTEST.COM'} at 127.0.0.1:49282)
I20260926 15:03:15.782316 15906 tablet_copy_service.cc:161] P c93ba0d1fefe4605b9d7f761e1e06f96: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b9febb6a22304d1f9376e29539155b4b at {username='oryx', principal='oryx/127.9.123.124@KRBTEST.COM'} at 127.0.0.1:49282: session id = b9febb6a22304d1f9376e29539155b4b-00000000000000000000000000000000
I20260926 15:03:15.786192 15906 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96: Tablet Copy: opened 0 blocks and 1 log segments
I20260926 15:03:15.788082 16447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260926 15:03:15.795591 16447 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b: tablet copy: Starting download of 0 data blocks...
I20260926 15:03:15.795917 16447 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b: tablet copy: Starting download of 1 WAL segments...
I20260926 15:03:15.798561 16447 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260926 15:03:15.803133 15906 tablet_copy_service.cc:342] P c93ba0d1fefe4605b9d7f761e1e06f96: Request end of tablet copy session b9febb6a22304d1f9376e29539155b4b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.123.124@KRBTEST.COM'} at 127.0.0.1:49282
I20260926 15:03:15.803289 15906 tablet_copy_service.cc:433] P c93ba0d1fefe4605b9d7f761e1e06f96: ending tablet copy session b9febb6a22304d1f9376e29539155b4b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b9febb6a22304d1f9376e29539155b4b
I20260926 15:03:15.816159 16447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:15.819082 16646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:15.825587 16647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:15.830123 16649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:15.836133 16447 server_base.cc:1034] running on GCE node
I20260926 15:03:15.836620 16447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:15.837395 16447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:15.838617 16447 hybrid_clock.cc:648] HybridClock initialized: now 1790434995838567 us; error 38 us; skew 500 ppm
I20260926 15:03:15.838861 16447 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:15 dist-test-slave-7153 krb5kdc[15841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434995, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:15.841017 16447 init.cc:377] Logged in from keytab as oryx/127.9.123.124@KRBTEST.COM (short username oryx)
I20260926 15:03:15.846148 16447 webserver.cc:460] Webserver started at http://127.9.123.124:34861/ using document root <none> and password file <none>
I20260926 15:03:15.846478 16447 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:15.846586 16447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:15.850322 16447 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:03:15.852380 16656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:15.852823 16447 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260926 15:03:15.852989 16447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/wal
uuid: "b9febb6a22304d1f9376e29539155b4b"
format_stamp: "Formatted at 2026-09-26 15:03:15 on dist-test-slave-7153"
server_key: "83fc37e03f7ff376470e002d1bb7324d"
server_key_iv: "7fde51f017b4894ce0e96c9e8b438276"
server_key_version: "encryptionkey@0"
I20260926 15:03:15.853255 16447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:15.871608 16447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:15.874522 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 15947
I20260926 15:03:15.875756 16447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:15.890973 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:46053
--local_ip_for_outbound_sockets=127.9.123.65
--tserver_master_addrs=127.9.123.126:32823
--webserver_port=38755
--webserver_interface=127.9.123.65
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.65.keytab
--principal=oryx/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:41353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.123.126:32823,127.9.123.125:36901
--tserver_master_addrs=127.9.123.126:32823,127.9.123.125:36901,127.9.123.124:45251 with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:03:15.912024 16447 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.124:45251
I20260926 15:03:15.912050 16708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.124:45251 every 8 connection(s)
I20260926 15:03:15.913072 16447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/master-2/data/info.pb
I20260926 15:03:15.918480 16709 sys_catalog.cc:263] Verifying existing consensus state
I20260926 15:03:15.922541 16709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b: Bootstrap starting.
I20260926 15:03:15.940385 16709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 15:03:15.940922 16709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b: Bootstrap complete.
I20260926 15:03:15.942109 16709 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: NON_VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: true } }
I20260926 15:03:15.942425 16709 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b9febb6a22304d1f9376e29539155b4b, State: Initialized, Role: LEARNER
I20260926 15:03:15.942715 16709 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: NON_VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: true } }
I20260926 15:03:15.943511 16712 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: NON_VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: true } } }
I20260926 15:03:15.943933 16712 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [sys.catalog]: This master's current role is: LEARNER
I20260926 15:03:15.943784 16709 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:03:15.960687 16723 catalog_manager.cc:1259] Loaded cluster ID: d728c8a2cf5643cc8a1ea2d38b1a354c
I20260926 15:03:15.960809 16723 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b: loading cluster ID for follower catalog manager: success
I20260926 15:03:15.964293 16723 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b: acquiring CA information for follower catalog manager: success
I20260926 15:03:15.966652 16723 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 15:03:16.134177 16607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:16.127613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:45251 (local address 127.0.0.1:47236)
0926 15:03:16.127792 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:16.127814 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:16.127980 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:16.128808 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:16.128829 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:16.128857 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:16.129013 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:16.129021 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.130739 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.130743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:16.132059 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:16.132075 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.133587 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.133591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:16.133746 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:16.133775 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 15:03:16.133847 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 15:03:16.133955 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260926 15:03:16.134326 16724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:16.127716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47236 (local address 127.9.123.124:45251)
0926 15:03:16.128458 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:16.128465 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:16.128487 (+ 22us) server_negotiation.cc:407] Connection header received
0926 15:03:16.128565 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:16.128570 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:16.128661 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:16.128793 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:16.129199 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.130612 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.132203 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.133442 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.133561 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:16.133767 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:16.133871 (+ 104us) server_negotiation.cc:299] Negotiation successful
0926 15:03:16.133990 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":459,"threads_started":1}
I20260926 15:03:16.135206 16684 raft_consensus.cc:1203] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260926 15:03:16.135778 16712 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c93ba0d1fefe4605b9d7f761e1e06f96. Latest consensus state: current_term: 1 leader_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: NON_VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: true } } }
I20260926 15:03:16.136049 16712 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [sys.catalog]: This master's current role is: LEARNER
W20260926 15:03:16.238948 16679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:16.239236 16679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:16.239351 16679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:16.247237 16679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:16.247367 16679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:16.247514 16679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:03:16.257670 16679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:41353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.65.keytab
--principal=oryx/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:46053
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=38755
--tserver_master_addrs=127.9.123.126:32823,127.9.123.125:36901,127.9.123.124:45251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:16.259043 16679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:16.260393 16679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:16.273809 16732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:16.273929 16730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:16.273830 16729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:16.274523 16679 server_base.cc:1034] running on GCE node
I20260926 15:03:16.275063 16679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:16.275920 16679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:16.277143 16679 hybrid_clock.cc:648] HybridClock initialized: now 1790434996277058 us; error 48 us; skew 500 ppm
I20260926 15:03:16.277487 16679 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:16 dist-test-slave-7153 krb5kdc[15841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434996, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:16.280462 16679 init.cc:377] Logged in from keytab as oryx/127.9.123.65@KRBTEST.COM (short username oryx)
I20260926 15:03:16.281896 16679 webserver.cc:460] Webserver started at http://127.9.123.65:38755/ using document root <none> and password file <none>
I20260926 15:03:16.282582 16679 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:16.282706 16679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:16.287946 16679 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260926 15:03:16.291098 16739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:16.292399 16679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260926 15:03:16.292547 16679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "fedeebde7a6744c69356ec6652b19d3d"
format_stamp: "Formatted at 2026-09-26 15:03:13 on dist-test-slave-7153"
server_key: "82813239964d21a0b7ef730f7ffaa914"
server_key_iv: "04b0dcda9ca9f6c8d97e0f32e6e9eb32"
server_key_version: "encryptionkey@0"
I20260926 15:03:16.293538 16679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:16.306975 16679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:16.311406 16679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:16.311663 16679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:16.312671 16679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:16.314153 16679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:16.314253 16679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:16.314355 16679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:16.314419 16679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:16.346345 16679 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:46053
I20260926 15:03:16.346391 16851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:46053 every 8 connection(s)
I20260926 15:03:16.347776 16679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:03:16.352365 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 16679
I20260926 15:03:16.352718 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 16081
Sep 26 15:03:16 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434996, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM
Sep 26 15:03:16 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.123.66@KRBTEST.COM
Sep 26 15:03:16 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434996, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.125@KRBTEST.COM
Sep 26 15:03:16 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434996, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.124@KRBTEST.COM
I20260926 15:03:16.417053 16862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:16.354458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36310 (local address 127.9.123.66:43577)
0926 15:03:16.388491 (+ 34033us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:16.388503 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:16.390840 (+ 2337us) server_negotiation.cc:407] Connection header received
0926 15:03:16.390935 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:16.390946 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:16.391276 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:16.391472 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:16.398403 (+ 6931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.400001 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.401903 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.402762 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.402861 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:16.409738 (+ 6877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:16.409756 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:16.409776 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:16.409821 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:16.413526 (+ 3705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:16.414433 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:16.414443 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:16.414446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:16.414615 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:16.415020 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:16.415026 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:16.415029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:16.415492 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:16.415755 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:16.416284 (+ 529us) server_negotiation.cc:299] Negotiation successful
0926 15:03:16.416587 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33790,"thread_start_us":36103,"threads_started":1}
I20260926 15:03:16.417599 16724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:16.350541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:57855 (local address 127.9.123.124:45251)
0926 15:03:16.350981 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:16.350989 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:16.397412 (+ 46423us) server_negotiation.cc:407] Connection header received
0926 15:03:16.397589 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:16.397595 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:16.397686 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:16.397778 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:16.399432 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.400417 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.403561 (+ 3144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.404179 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.404280 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:16.410585 (+ 6305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:16.410593 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:16.410597 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:16.410654 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:16.413780 (+ 3126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:16.414972 (+ 1192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:16.414977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:16.414980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:16.415122 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:16.415589 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:16.415594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:16.415597 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:16.415944 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:16.416087 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:16.417011 (+ 924us) server_negotiation.cc:299] Negotiation successful
0926 15:03:16.417288 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290}
I20260926 15:03:16.418365 16860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:16.351927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:53395 (local address 127.9.123.126:32823)
0926 15:03:16.396101 (+ 44174us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:16.396109 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:16.396131 (+ 22us) server_negotiation.cc:407] Connection header received
0926 15:03:16.396196 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:16.396202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:16.396292 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:16.396407 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:16.399247 (+ 2840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.400320 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.403554 (+ 3234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.404180 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.404273 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:16.409188 (+ 4915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:16.409195 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:16.409199 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:16.409244 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:16.413443 (+ 4199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:16.414935 (+ 1492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:16.414940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:16.414943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:16.415092 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:16.415625 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:16.415630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:16.415633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:16.415938 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:16.416074 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:16.417794 (+ 1720us) server_negotiation.cc:299] Negotiation successful
0926 15:03:16.418015 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44017,"thread_start_us":43892,"threads_started":1}
I20260926 15:03:16.418797 16858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:16.351149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.9.123.65:53395)
0926 15:03:16.392432 (+ 41283us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:16.392507 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:16.395270 (+ 2763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:16.396984 (+ 1714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:16.397000 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:16.397967 (+ 967us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:16.399015 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:16.399043 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.400510 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.400516 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:16.403383 (+ 2867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:16.403398 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.404629 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.404636 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:16.405046 (+ 410us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:16.405877 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:16.405919 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:16.408927 (+ 3008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:16.414210 (+ 5283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:16.414214 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:16.414217 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:16.414825 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:16.415208 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:16.415212 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:16.415215 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:16.415475 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:16.416074 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:16.416083 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:16.416553 (+ 470us) client_negotiation.cc:769] Sending connection context
0926 15:03:16.416864 (+ 311us) client_negotiation.cc:240] Negotiation successful
0926 15:03:16.417458 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40908,"mutex_wait_us":288,"spinlock_wait_cycles":104320,"thread_start_us":46722,"threads_started":1}
I20260926 15:03:16.419441 16204 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:16.419791 16856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:16.350599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:59025 (local address 127.9.123.125:36901)
0926 15:03:16.351117 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:16.351123 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:16.398546 (+ 47423us) server_negotiation.cc:407] Connection header received
0926 15:03:16.398800 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:16.398806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:16.398897 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:16.398971 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:16.400189 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.401228 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.403697 (+ 2469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.404271 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.404350 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:16.410575 (+ 6225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:16.410582 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:16.410587 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:16.410629 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:16.414006 (+ 3377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:16.416155 (+ 2149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:16.416160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:16.416164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:16.416307 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:16.416852 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:16.416858 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:16.416861 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:16.417486 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:16.417635 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:16.419045 (+ 1410us) server_negotiation.cc:299] Negotiation successful
0926 15:03:16.419496 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":367,"threads_started":1}
I20260926 15:03:16.420004 16857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:16.351430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:36901 (local address 127.9.123.65:59025)
0926 15:03:16.398429 (+ 46999us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:16.398447 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:16.398654 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:16.399188 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:16.399192 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:16.399741 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:16.399994 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:16.400005 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.401427 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.401432 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:16.403556 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:16.403566 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.404629 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.404636 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:16.405041 (+ 405us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:16.405877 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:16.405919 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:16.410341 (+ 4422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:16.414137 (+ 3796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:16.414142 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:16.414145 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:16.415444 (+ 1299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:16.416485 (+ 1041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:16.416489 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:16.416493 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:16.416723 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:16.418608 (+ 1885us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:16.418613 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:16.418755 (+ 142us) client_negotiation.cc:769] Sending connection context
0926 15:03:16.418924 (+ 169us) client_negotiation.cc:240] Negotiation successful
0926 15:03:16.419200 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46886,"mutex_wait_us":252,"spinlock_wait_cycles":62208,"thread_start_us":46078,"threads_started":1}
I20260926 15:03:16.420557 16852 heartbeater.cc:347] Connected to a master server at 127.9.123.126:32823
I20260926 15:03:16.420887 16852 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:16.420969 16854 heartbeater.cc:347] Connected to a master server at 127.9.123.125:36901
I20260926 15:03:16.421075 16854 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:16.422276 16852 heartbeater.cc:511] Master 127.9.123.126:32823 requested a full tablet report, sending...
I20260926 15:03:16.422295 16854 heartbeater.cc:511] Master 127.9.123.125:36901 requested a full tablet report, sending...
I20260926 15:03:16.424149 15886 ts_manager.cc:194] Re-registered known tserver with Master: fedeebde7a6744c69356ec6652b19d3d (127.9.123.65:46053)
I20260926 15:03:16.424961 15886 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.123.65@KRBTEST.COM'} at 127.9.123.65:53395
I20260926 15:03:16.425304 16484 ts_manager.cc:194] Registered new tserver with Master: fedeebde7a6744c69356ec6652b19d3d (127.9.123.65:46053)
I20260926 15:03:16.426151 16859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:16.351421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:45251 (local address 127.9.123.65:57855)
0926 15:03:16.397287 (+ 45866us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:16.397322 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:16.397462 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:16.398208 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:16.398213 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:16.398944 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:16.399243 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:16.399255 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.400586 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.400591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:16.403391 (+ 2800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:16.403405 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.404691 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.404700 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:16.405036 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:16.405973 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:16.406000 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:16.410352 (+ 4352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:16.413995 (+ 3643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:16.414005 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:16.414027 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:16.414821 (+ 794us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:16.415302 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:16.415308 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:16.415312 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:16.415474 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:16.416203 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:16.416207 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:16.416549 (+ 342us) client_negotiation.cc:769] Sending connection context
0926 15:03:16.425378 (+ 8829us) client_negotiation.cc:240] Negotiation successful
0926 15:03:16.425716 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45724,"mutex_wait_us":97,"spinlock_wait_cycles":14848,"thread_start_us":40453,"threads_started":1}
I20260926 15:03:16.427196 16853 heartbeater.cc:347] Connected to a master server at 127.9.123.124:45251
I20260926 15:03:16.427348 16853 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:16.427769 16853 heartbeater.cc:511] Master 127.9.123.124:45251 requested a full tablet report, sending...
I20260926 15:03:16.429329 16673 ts_manager.cc:194] Registered new tserver with Master: fedeebde7a6744c69356ec6652b19d3d (127.9.123.65:46053)
I20260926 15:03:16.514829 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 16081
I20260926 15:03:16.528152 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:43577
--local_ip_for_outbound_sockets=127.9.123.66
--tserver_master_addrs=127.9.123.126:32823
--webserver_port=41585
--webserver_interface=127.9.123.66
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.66.keytab
--principal=oryx/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:41353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.123.126:32823,127.9.123.125:36901
--tserver_master_addrs=127.9.123.126:32823,127.9.123.125:36901,127.9.123.124:45251 with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:03:16.545042 16634 sys_catalog.cc:455] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: NON_VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: true } } }
I20260926 15:03:16.545804 16634 sys_catalog.cc:458] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:16.546665 16866 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96: attempting to promote NON_VOTER b9febb6a22304d1f9376e29539155b4b to VOTER
I20260926 15:03:16.547240 16866 consensus_queue.cc:237] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: false } }
I20260926 15:03:16.548943 16494 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 1 FOLLOWER]: Refusing update from remote peer c93ba0d1fefe4605b9d7f761e1e06f96: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260926 15:03:16.548966 16684 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [term 1 LEARNER]: Refusing update from remote peer c93ba0d1fefe4605b9d7f761e1e06f96: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260926 15:03:16.549525 16865 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260926 15:03:16.549908 16866 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260926 15:03:16.553167 16865 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, b9febb6a22304d1f9376e29539155b4b (127.9.123.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: false } } }
I20260926 15:03:16.554962 16684 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, b9febb6a22304d1f9376e29539155b4b (127.9.123.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: false } } }
I20260926 15:03:16.556077 16865 sys_catalog.cc:455] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: false } } }
I20260926 15:03:16.556392 16865 sys_catalog.cc:458] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:16.556638 16634 sys_catalog.cc:455] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: false } } }
I20260926 15:03:16.556880 16634 sys_catalog.cc:458] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:16.556813 16866 sys_catalog.cc:455] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: false } } }
I20260926 15:03:16.557035 16866 sys_catalog.cc:458] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:16.557704 16494 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, b9febb6a22304d1f9376e29539155b4b (127.9.123.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: false } } }
I20260926 15:03:16.560561 16712 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: false } } }
I20260926 15:03:16.560797 16712 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:03:16.560904 16869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: false } } }
I20260926 15:03:16.561386 16869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [sys.catalog]: This master's current role is: FOLLOWER
W20260926 15:03:16.862048 16864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:16.862339 16864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:16.862458 16864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:16.870024 16864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:16.870138 16864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:16.870303 16864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:03:16.880173 16864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:41353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.66.keytab
--principal=oryx/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:43577
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=41585
--tserver_master_addrs=127.9.123.126:32823,127.9.123.125:36901,127.9.123.124:45251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:16.881671 16864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:16.883068 16864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:16.896057 16875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:16.896093 16877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:16.896118 16874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:16.897241 16864 server_base.cc:1034] running on GCE node
I20260926 15:03:16.897929 16864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:16.898890 16864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:16.900102 16864 hybrid_clock.cc:648] HybridClock initialized: now 1790434996900052 us; error 45 us; skew 500 ppm
I20260926 15:03:16.900421 16864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:16 dist-test-slave-7153 krb5kdc[15841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434996, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:16.903529 16864 init.cc:377] Logged in from keytab as oryx/127.9.123.66@KRBTEST.COM (short username oryx)
I20260926 15:03:16.905011 16864 webserver.cc:460] Webserver started at http://127.9.123.66:41585/ using document root <none> and password file <none>
I20260926 15:03:16.905726 16864 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:16.905853 16864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:16.911326 16864 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 15:03:16.914435 16884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:16.916090 16864 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:03:16.916253 16864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "7b83c187309b40c8ae16f8149b7ad171"
format_stamp: "Formatted at 2026-09-26 15:03:13 on dist-test-slave-7153"
server_key: "e40d5c0f1b2a6ae6109ce84774014a7f"
server_key_iv: "7b1eb3e2759e42226be55feaf4e67d9c"
server_key_version: "encryptionkey@0"
I20260926 15:03:16.917416 16864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:16.935957 16864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:16.940596 16864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:16.940874 16864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:16.941890 16864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:16.943292 16864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:16.943388 16864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:16.943499 16864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:16.943575 16864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:16.973220 16864 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:43577
I20260926 15:03:16.973260 16996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:43577 every 8 connection(s)
I20260926 15:03:16.974673 16864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:03:16.979056 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 16864
I20260926 15:03:16.979369 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 16214
Sep 26 15:03:17 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.123.67@KRBTEST.COM
Sep 26 15:03:17 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434996, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM
Sep 26 15:03:17 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434996, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.124@KRBTEST.COM
Sep 26 15:03:17 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434996, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.125@KRBTEST.COM
I20260926 15:03:17.012080 17003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:16.977705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:51897 (local address 127.9.123.125:36901)
0926 15:03:16.991570 (+ 13865us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:16.991576 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:16.991598 (+ 22us) server_negotiation.cc:407] Connection header received
0926 15:03:16.992232 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:16.992238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:16.992322 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:16.992425 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:16.995022 (+ 2597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.996108 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.998961 (+ 2853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.000076 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.000170 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.005047 (+ 4877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:17.005053 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:17.005058 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:17.005134 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:17.008386 (+ 3252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:17.009331 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:17.009343 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:17.009347 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:17.009497 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:17.010024 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:17.010029 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:17.010032 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:17.010412 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:17.010553 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:17.011494 (+ 941us) server_negotiation.cc:299] Negotiation successful
0926 15:03:17.011730 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13732,"thread_start_us":13901,"threads_started":1}
I20260926 15:03:17.012765 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:16.977410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:45279 (local address 127.9.123.124:45251)
0926 15:03:16.992254 (+ 14844us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:16.992260 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:16.992278 (+ 18us) server_negotiation.cc:407] Connection header received
0926 15:03:16.992328 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:16.992332 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:16.992400 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:16.992488 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:16.995022 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.996152 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.998961 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.999548 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.999638 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.004758 (+ 5120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:17.004765 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:17.004771 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:17.004812 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:17.008179 (+ 3367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:17.009309 (+ 1130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:17.009314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:17.009317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:17.009468 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:17.010003 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:17.010008 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:17.010012 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:17.010367 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:17.010497 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:17.011905 (+ 1408us) server_negotiation.cc:299] Negotiation successful
0926 15:03:17.012114 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14734,"thread_start_us":14614,"threads_started":1}
I20260926 15:03:17.013773 17006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:16.980842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34846 (local address 127.9.123.67:40451)
0926 15:03:16.987122 (+ 6280us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:16.987131 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:16.987163 (+ 32us) server_negotiation.cc:407] Connection header received
0926 15:03:16.987237 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:16.987245 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:16.987518 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:16.987665 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:16.992341 (+ 4676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.993950 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.995769 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.996601 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.996734 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.005183 (+ 8449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:17.005202 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:17.005231 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:17.005274 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:17.009353 (+ 4079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:17.010275 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:17.010285 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:17.010289 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:17.010445 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:17.010850 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:17.010855 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:17.010858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:17.011574 (+ 716us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:17.011891 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:17.013197 (+ 1306us) server_negotiation.cc:299] Negotiation successful
0926 15:03:17.013436 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6078,"thread_start_us":7019,"threads_started":1}
I20260926 15:03:17.013953 17007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:16.981134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:36901 (local address 127.9.123.66:51897)
0926 15:03:16.988338 (+ 7204us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:16.988358 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:16.991678 (+ 3320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:16.992786 (+ 1108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:16.992801 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:16.993791 (+ 990us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:16.994814 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:16.994841 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.996305 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.996312 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:16.998793 (+ 2481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:16.998809 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.000452 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.000458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:17.000649 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.001073 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:17.001091 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:17.004827 (+ 3736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:17.008564 (+ 3737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:17.008569 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:17.008573 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:17.009190 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:17.009625 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:17.009628 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:17.009631 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:17.009890 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:17.010618 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:17.010622 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:17.011054 (+ 432us) client_negotiation.cc:769] Sending connection context
0926 15:03:17.012342 (+ 1288us) client_negotiation.cc:240] Negotiation successful
0926 15:03:17.012638 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7065,"mutex_wait_us":249,"spinlock_wait_cycles":141440,"thread_start_us":6902,"threads_started":1}
I20260926 15:03:17.014407 17008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:16.981613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:45251 (local address 127.9.123.66:45279)
0926 15:03:16.988870 (+ 7257us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:16.988885 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:16.991573 (+ 2688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:16.992786 (+ 1213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:16.992801 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:16.993814 (+ 1013us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:16.994814 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:16.994841 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.996303 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.996308 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:16.998823 (+ 2515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:16.998832 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.999918 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.999924 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:17.000203 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.000956 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:17.000990 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:17.004198 (+ 3208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:17.008374 (+ 4176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:17.008385 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:17.008405 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:17.009154 (+ 749us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:17.009592 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:17.009597 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:17.009600 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:17.009857 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:17.010574 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:17.010583 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:17.011055 (+ 472us) client_negotiation.cc:769] Sending connection context
0926 15:03:17.011387 (+ 332us) client_negotiation.cc:240] Negotiation successful
0926 15:03:17.014016 (+ 2629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7145,"mutex_wait_us":2574,"spinlock_wait_cycles":45056,"thread_start_us":6996,"threads_started":1}
I20260926 15:03:17.015095 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:16.977469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:55907 (local address 127.9.123.126:32823)
0926 15:03:16.992035 (+ 14566us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:16.992041 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:16.992061 (+ 20us) server_negotiation.cc:407] Connection header received
0926 15:03:16.992120 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:16.992125 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:16.992211 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:16.992311 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:16.995026 (+ 2715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.996012 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.999009 (+ 2997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.000143 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.000283 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.005275 (+ 4992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:17.005282 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:17.005287 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:17.005328 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:17.009473 (+ 4145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:17.010286 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:17.010292 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:17.010296 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:17.010436 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:17.010844 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:17.010848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:17.010851 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:17.011172 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:17.011304 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:17.012319 (+ 1015us) server_negotiation.cc:299] Negotiation successful
0926 15:03:17.012666 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14453,"thread_start_us":14330,"threads_started":1}
I20260926 15:03:17.015245 16337 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:17.015370 17004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:16.978209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.9.123.66:55907)
0926 15:03:16.987238 (+ 9029us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:16.987303 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:16.991103 (+ 3800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:16.992786 (+ 1683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:16.992800 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:16.993773 (+ 973us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:16.994826 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:16.994840 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:16.996197 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:16.996203 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:16.998824 (+ 2621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:16.998840 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.000549 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.000557 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:17.000721 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.001258 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:17.001280 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:17.004069 (+ 2789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:17.009618 (+ 5549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:17.009622 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:17.009626 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:17.010150 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:17.010553 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:17.010558 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:17.010561 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:17.010729 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:17.011358 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:17.011362 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:17.011501 (+ 139us) client_negotiation.cc:769] Sending connection context
0926 15:03:17.011611 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 15:03:17.012002 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8648,"mutex_wait_us":466,"thread_start_us":8410,"threads_started":1}
I20260926 15:03:17.015657 16998 heartbeater.cc:347] Connected to a master server at 127.9.123.124:45251
I20260926 15:03:17.015972 16998 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:17.016636 16998 heartbeater.cc:511] Master 127.9.123.124:45251 requested a full tablet report, sending...
I20260926 15:03:17.015683 16999 heartbeater.cc:347] Connected to a master server at 127.9.123.125:36901
I20260926 15:03:17.017423 16999 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:17.017556 16997 heartbeater.cc:347] Connected to a master server at 127.9.123.126:32823
I20260926 15:03:17.017669 16997 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:17.017935 16999 heartbeater.cc:511] Master 127.9.123.125:36901 requested a full tablet report, sending...
I20260926 15:03:17.018065 16997 heartbeater.cc:511] Master 127.9.123.126:32823 requested a full tablet report, sending...
I20260926 15:03:17.019042 16484 ts_manager.cc:194] Registered new tserver with Master: 7b83c187309b40c8ae16f8149b7ad171 (127.9.123.66:43577)
I20260926 15:03:17.019392 16673 ts_manager.cc:194] Registered new tserver with Master: 7b83c187309b40c8ae16f8149b7ad171 (127.9.123.66:43577)
I20260926 15:03:17.020198 15886 ts_manager.cc:194] Re-registered known tserver with Master: 7b83c187309b40c8ae16f8149b7ad171 (127.9.123.66:43577)
I20260926 15:03:17.021003 15886 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.123.66@KRBTEST.COM'} at 127.9.123.66:55907
I20260926 15:03:17.103256 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 16214
I20260926 15:03:17.117026 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:40451
--local_ip_for_outbound_sockets=127.9.123.67
--tserver_master_addrs=127.9.123.126:32823
--webserver_port=46341
--webserver_interface=127.9.123.67
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.67.keytab
--principal=oryx/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:41353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.123.126:32823,127.9.123.125:36901
--tserver_master_addrs=127.9.123.126:32823,127.9.123.125:36901,127.9.123.124:45251 with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:03:17.428282 16852 heartbeater.cc:503] Master 127.9.123.126:32823 was elected leader, sending a full tablet report...
W20260926 15:03:17.456735 17010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:17.457024 17010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:17.457185 17010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:17.464490 17010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:17.464617 17010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:17.464771 17010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:03:17.474864 17010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:41353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.67.keytab
--principal=oryx/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:40451
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=46341
--tserver_master_addrs=127.9.123.126:32823,127.9.123.125:36901,127.9.123.124:45251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:17.476246 17010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:17.477588 17010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:17.491351 17018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:17.491351 17015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:17.491784 17010 server_base.cc:1034] running on GCE node
W20260926 15:03:17.491858 17016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:17.492614 17010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:17.493594 17010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:17.494846 17010 hybrid_clock.cc:648] HybridClock initialized: now 1790434997494776 us; error 68 us; skew 500 ppm
I20260926 15:03:17.495169 17010 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:17 dist-test-slave-7153 krb5kdc[15841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434997, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:17.498325 17010 init.cc:377] Logged in from keytab as oryx/127.9.123.67@KRBTEST.COM (short username oryx)
I20260926 15:03:17.499681 17010 webserver.cc:460] Webserver started at http://127.9.123.67:46341/ using document root <none> and password file <none>
I20260926 15:03:17.500329 17010 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:17.500437 17010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:17.505920 17010 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 15:03:17.508762 17025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:17.510383 17010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 15:03:17.510548 17010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "b0a766fb9dc74a42967c0463588f3257"
format_stamp: "Formatted at 2026-09-26 15:03:14 on dist-test-slave-7153"
server_key: "6ca2cbff094fd35d0c0f8982890b93f9"
server_key_iv: "caaac7fd17ec82f0b39aa0bffd539bc9"
server_key_version: "encryptionkey@0"
I20260926 15:03:17.511677 17010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:17.553503 17010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:17.557883 17010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:17.558140 17010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:17.559113 17010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:17.560456 17010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:17.560551 17010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:17.560688 17010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:17.560752 17010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:17.590117 17010 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:40451
I20260926 15:03:17.590171 17138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:40451 every 8 connection(s)
I20260926 15:03:17.591511 17010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:03:17.597517 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 17010
Sep 26 15:03:17 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434997, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.124@KRBTEST.COM
Sep 26 15:03:17 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434997, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.125@KRBTEST.COM
I20260926 15:03:17.660898 17156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:17.638325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49556 (local address 127.9.123.125:36901)
0926 15:03:17.638939 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:17.638945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:17.639596 (+ 651us) server_negotiation.cc:407] Connection header received
0926 15:03:17.639884 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:17.639889 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:17.639964 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:17.640039 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:17.642688 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.643729 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.646533 (+ 2804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.647458 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.647553 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.652895 (+ 5342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:17.652903 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:17.652908 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:17.652956 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:17.656742 (+ 3786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:17.657733 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:17.657740 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:17.657744 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:17.657974 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:17.658468 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:17.658473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:17.658477 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:17.658843 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:17.659140 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:17.660238 (+ 1098us) server_negotiation.cc:299] Negotiation successful
0926 15:03:17.660543 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":737,"threads_started":1}
Sep 26 15:03:17 dist-test-slave-7153 krb5kdc[15841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790434997, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM
I20260926 15:03:17.667412 17154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:17.637824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47246 (local address 127.9.123.124:45251)
0926 15:03:17.641263 (+ 3439us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:17.641270 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:17.646429 (+ 5159us) server_negotiation.cc:407] Connection header received
0926 15:03:17.649190 (+ 2761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:17.649195 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:17.649268 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:17.649345 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:17.651913 (+ 2568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.653149 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.655988 (+ 2839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.656612 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.656726 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.660691 (+ 3965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:17.660698 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:17.660704 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:17.660747 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:17.664060 (+ 3313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:17.664823 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:17.664828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:17.664831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:17.664995 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:17.665447 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:17.665451 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:17.665455 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:17.665841 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:17.666116 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:17.666689 (+ 573us) server_negotiation.cc:299] Negotiation successful
0926 15:03:17.666983 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":216,"threads_started":1}
I20260926 15:03:17.672770 17144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:17.594465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:57891 (local address 127.9.123.126:32823)
0926 15:03:17.635293 (+ 40828us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:17.635300 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:17.635322 (+ 22us) server_negotiation.cc:407] Connection header received
0926 15:03:17.636950 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:17.636956 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:17.637039 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:17.637182 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:17.640141 (+ 2959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.641359 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.644426 (+ 3067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.645086 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.645219 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.664000 (+ 18781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:17.664008 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:17.664014 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:17.664064 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:17.667530 (+ 3466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:17.670151 (+ 2621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:17.670157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:17.670160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:17.670319 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:17.670753 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:17.670759 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:17.670762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:17.671097 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:17.671248 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:17.672169 (+ 921us) server_negotiation.cc:299] Negotiation successful
0926 15:03:17.672418 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40707,"thread_start_us":40852,"threads_started":1}
I20260926 15:03:17.672847 17145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:17.595249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:41373 (local address 127.9.123.124:45251)
0926 15:03:17.635725 (+ 40476us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:17.635729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:17.635746 (+ 17us) server_negotiation.cc:407] Connection header received
0926 15:03:17.637079 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:17.637085 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:17.637184 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:17.637286 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:17.640781 (+ 3495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.641881 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.646980 (+ 5099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.647613 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.647702 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.664731 (+ 17029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:17.664738 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:17.664743 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:17.664780 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:17.668087 (+ 3307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:17.669463 (+ 1376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:17.669468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:17.669471 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:17.669620 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:17.670762 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:17.670766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:17.670770 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:17.671031 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:17.671120 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:17.672266 (+ 1146us) server_negotiation.cc:299] Negotiation successful
0926 15:03:17.672481 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40353,"thread_start_us":42097,"threads_started":1}
I20260926 15:03:17.673842 17143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:17.594468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:57075 (local address 127.9.123.125:36901)
0926 15:03:17.635426 (+ 40958us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:17.635432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:17.635452 (+ 20us) server_negotiation.cc:407] Connection header received
0926 15:03:17.646958 (+ 11506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:17.646964 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:17.647053 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:17.648400 (+ 1347us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:17.648653 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.649861 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.651910 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.652528 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.653261 (+ 733us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.659216 (+ 5955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:17.659222 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:17.659226 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:17.659263 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:17.668440 (+ 9177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:17.669489 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:17.669494 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:17.669497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:17.669625 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:17.670102 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:17.670105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:17.670107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:17.672182 (+ 2075us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:17.673076 (+ 894us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:17.673240 (+ 164us) server_negotiation.cc:299] Negotiation successful
0926 15:03:17.673454 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40852,"thread_start_us":40759,"threads_started":1}
I20260926 15:03:17.674120 17146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:17.595059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:36901 (local address 127.9.123.67:57075)
0926 15:03:17.634999 (+ 39940us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:17.635019 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:17.636623 (+ 1604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:17.647259 (+ 10636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:17.647265 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:17.647882 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:17.648244 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:17.648257 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.650036 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.650043 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:17.651731 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:17.651747 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.652733 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.652740 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:17.652916 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.653517 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:17.653540 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:17.658893 (+ 5353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:17.668603 (+ 9710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:17.668607 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:17.668610 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:17.669395 (+ 785us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:17.669743 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:17.669747 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:17.669750 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:17.670003 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:17.672325 (+ 2322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:17.672330 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:17.672474 (+ 144us) client_negotiation.cc:769] Sending connection context
0926 15:03:17.672612 (+ 138us) client_negotiation.cc:240] Negotiation successful
0926 15:03:17.672872 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39769,"mutex_wait_us":1744,"thread_start_us":32256,"threads_started":1}
I20260926 15:03:17.674222 17147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:17.595724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:45251 (local address 127.9.123.67:41373)
0926 15:03:17.633789 (+ 38065us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:17.633814 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:17.636610 (+ 2796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:17.637510 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:17.637514 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:17.640335 (+ 2821us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:17.640592 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:17.640603 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.644496 (+ 3893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.644501 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:17.645931 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:17.645948 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.648485 (+ 2537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.648491 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:17.648674 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.654592 (+ 5918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:17.654615 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:17.664499 (+ 9884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:17.668251 (+ 3752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:17.668263 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:17.668287 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:17.669329 (+ 1042us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:17.670448 (+ 1119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:17.670452 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:17.670455 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:17.670644 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:17.671212 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:17.671221 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:17.671755 (+ 534us) client_negotiation.cc:769] Sending connection context
0926 15:03:17.673435 (+ 1680us) client_negotiation.cc:240] Negotiation successful
0926 15:03:17.673713 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37891,"mutex_wait_us":59,"spinlock_wait_cycles":32128,"thread_start_us":39503,"threads_started":1}
I20260926 15:03:17.675647 17141 heartbeater.cc:347] Connected to a master server at 127.9.123.125:36901
I20260926 15:03:17.675971 17141 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:17.676051 17148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:17.595059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.9.123.67:57891)
0926 15:03:17.631515 (+ 36456us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:17.631586 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:17.636610 (+ 5024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:17.637466 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:17.637480 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:17.638916 (+ 1436us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:17.639959 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:17.639990 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.642069 (+ 2079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.642075 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:17.644288 (+ 2213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:17.644307 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.646203 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.646211 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:17.646519 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.648035 (+ 1516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:17.648078 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:17.663761 (+ 15683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:17.668251 (+ 4490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:17.668263 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:17.668287 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:17.669629 (+ 1342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:17.670426 (+ 797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:17.670431 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:17.670434 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:17.670609 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:17.671230 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:17.671233 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:17.671755 (+ 522us) client_negotiation.cc:769] Sending connection context
0926 15:03:17.672095 (+ 340us) client_negotiation.cc:240] Negotiation successful
0926 15:03:17.672648 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33891,"mutex_wait_us":33,"spinlock_wait_cycles":29312,"thread_start_us":39487,"threads_started":1}
I20260926 15:03:17.676918 17139 heartbeater.cc:347] Connected to a master server at 127.9.123.126:32823
I20260926 15:03:17.676940 17140 heartbeater.cc:347] Connected to a master server at 127.9.123.124:45251
I20260926 15:03:17.676975 17141 heartbeater.cc:511] Master 127.9.123.125:36901 requested a full tablet report, sending...
I20260926 15:03:17.677073 17139 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:17.677141 17140 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:17.677619 17139 heartbeater.cc:511] Master 127.9.123.126:32823 requested a full tablet report, sending...
I20260926 15:03:17.677668 17140 heartbeater.cc:511] Master 127.9.123.124:45251 requested a full tablet report, sending...
I20260926 15:03:17.678800 16673 ts_manager.cc:194] Registered new tserver with Master: b0a766fb9dc74a42967c0463588f3257 (127.9.123.67:40451)
I20260926 15:03:17.678797 16484 ts_manager.cc:194] Registered new tserver with Master: b0a766fb9dc74a42967c0463588f3257 (127.9.123.67:40451)
I20260926 15:03:17.679222 15886 ts_manager.cc:194] Re-registered known tserver with Master: b0a766fb9dc74a42967c0463588f3257 (127.9.123.67:40451)
I20260926 15:03:17.680186 15886 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.123.67@KRBTEST.COM'} at 127.9.123.67:57891
I20260926 15:03:17.685405 17157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:17.638792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49288 (local address 127.9.123.126:32823)
0926 15:03:17.669956 (+ 31164us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:17.669962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:17.669980 (+ 18us) server_negotiation.cc:407] Connection header received
0926 15:03:17.670044 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:17.670049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:17.670145 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:17.670254 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:17.671945 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.674139 (+ 2194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.675829 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.676530 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.676632 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.680020 (+ 3388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:17.680028 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:17.680032 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:17.680069 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:17.683018 (+ 2949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:17.683606 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:17.683610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:17.683612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:17.683762 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:17.684069 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:17.684071 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:17.684073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:17.684322 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:17.684484 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:17.684782 (+ 298us) server_negotiation.cc:299] Negotiation successful
0926 15:03:17.685007 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31045,"thread_start_us":26800,"threads_started":1}
I20260926 15:03:17.689291 15886 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:03:17.695984 15886 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:03:17.729508 17170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:17.716186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:46053 (local address 127.0.0.1:44122)
0926 15:03:17.716830 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:17.716849 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:17.717027 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:17.719866 (+ 2839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:17.719871 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:17.719899 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:17.720339 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:17.720351 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.723360 (+ 3009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.723366 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:17.726408 (+ 3042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:17.726427 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.728556 (+ 2129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.728564 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:17.728808 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.728850 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 15:03:17.728922 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 15:03:17.729090 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":356,"threads_started":1}
I20260926 15:03:17.730976 17171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:17.716485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44122 (local address 127.9.123.65:46053)
0926 15:03:17.719208 (+ 2723us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:17.719217 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:17.719245 (+ 28us) server_negotiation.cc:407] Connection header received
0926 15:03:17.719364 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:17.719372 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:17.719638 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:17.720568 (+ 930us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:17.720813 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.723174 (+ 2361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.726663 (+ 3489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.728271 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.730088 (+ 1817us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.730372 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:17.730454 (+ 82us) server_negotiation.cc:299] Negotiation successful
0926 15:03:17.730644 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2551,"thread_start_us":2837,"threads_started":1}
I20260926 15:03:17.731359 17172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:17.716740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:40451 (local address 127.0.0.1:34850)
0926 15:03:17.717273 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:17.717286 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:17.717366 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:17.718451 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:17.718455 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:17.718480 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:17.718850 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:17.718864 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.721838 (+ 2974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.721844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:17.726067 (+ 4223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:17.726086 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.728680 (+ 2594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.728687 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:17.728895 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.728933 (+ 38us) client_negotiation.cc:769] Sending connection context
0926 15:03:17.729022 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 15:03:17.730832 (+ 1810us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":5,"thread_start_us":741,"threads_started":1}
I20260926 15:03:17.732311 17173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:17.717321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34850 (local address 127.9.123.67:40451)
0926 15:03:17.717835 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:17.717842 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:17.717872 (+ 30us) server_negotiation.cc:407] Connection header received
0926 15:03:17.717950 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:17.717956 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:17.718265 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:17.718478 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:17.719043 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.721511 (+ 2468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.726289 (+ 4778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.728101 (+ 1812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.728249 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.728499 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:17.731747 (+ 3248us) server_negotiation.cc:299] Negotiation successful
0926 15:03:17.731953 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":671,"threads_started":1}
I20260926 15:03:17.735432 16787 tablet_service.cc:1467] Processing CreateTablet for tablet 27bf5cb34c7b4ef19f07522828f66ebf (DEFAULT_TABLE table=test-table [id=1f32dc15d1a6409e81fdb0d1b9061584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:03:17.736987 16787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27bf5cb34c7b4ef19f07522828f66ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:17.736840 17073 tablet_service.cc:1467] Processing CreateTablet for tablet 27bf5cb34c7b4ef19f07522828f66ebf (DEFAULT_TABLE table=test-table [id=1f32dc15d1a6409e81fdb0d1b9061584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:03:17.738217 17073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27bf5cb34c7b4ef19f07522828f66ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:17.742084 17170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:17.718983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:43577 (local address 127.0.0.1:36320)
0926 15:03:17.729794 (+ 10811us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:17.729816 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:17.729926 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:17.734041 (+ 4115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:17.734046 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:17.734078 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:17.734255 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:17.734264 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.738474 (+ 4210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.738481 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:17.739644 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:17.739658 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.741376 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.741382 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:17.741590 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.741623 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 15:03:17.741718 (+ 95us) client_negotiation.cc:240] Negotiation successful
0926 15:03:17.741866 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10681,"thread_start_us":14537,"threads_started":1}
I20260926 15:03:17.742741 17175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:17.719208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36320 (local address 127.9.123.66:43577)
0926 15:03:17.733044 (+ 13836us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:17.733054 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:17.733084 (+ 30us) server_negotiation.cc:407] Connection header received
0926 15:03:17.733506 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:17.733516 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:17.733781 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:17.737167 (+ 3386us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:17.737357 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.738335 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.739800 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.740715 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.741056 (+ 341us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.741291 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:17.742402 (+ 1111us) server_negotiation.cc:299] Negotiation successful
0926 15:03:17.742539 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13667,"thread_start_us":13903,"threads_started":1}
I20260926 15:03:17.745898 16932 tablet_service.cc:1467] Processing CreateTablet for tablet 27bf5cb34c7b4ef19f07522828f66ebf (DEFAULT_TABLE table=test-table [id=1f32dc15d1a6409e81fdb0d1b9061584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:03:17.747001 16932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27bf5cb34c7b4ef19f07522828f66ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:17.752754 17176 tablet_bootstrap.cc:492] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d: Bootstrap starting.
I20260926 15:03:17.754607 17177 tablet_bootstrap.cc:492] T 27bf5cb34c7b4ef19f07522828f66ebf P b0a766fb9dc74a42967c0463588f3257: Bootstrap starting.
I20260926 15:03:17.756932 17176 tablet_bootstrap.cc:654] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:17.757488 17177 tablet_bootstrap.cc:654] T 27bf5cb34c7b4ef19f07522828f66ebf P b0a766fb9dc74a42967c0463588f3257: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:17.758544 17177 log.cc:824] T 27bf5cb34c7b4ef19f07522828f66ebf P b0a766fb9dc74a42967c0463588f3257: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:17.758802 17176 log.cc:824] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:17.760857 17180 tablet_bootstrap.cc:492] T 27bf5cb34c7b4ef19f07522828f66ebf P 7b83c187309b40c8ae16f8149b7ad171: Bootstrap starting.
I20260926 15:03:17.762432 17176 tablet_bootstrap.cc:492] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d: No bootstrap required, opened a new log
I20260926 15:03:17.762701 17176 ts_tablet_manager.cc:1397] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260926 15:03:17.763664 17180 tablet_bootstrap.cc:654] T 27bf5cb34c7b4ef19f07522828f66ebf P 7b83c187309b40c8ae16f8149b7ad171: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:17.764561 17180 log.cc:824] T 27bf5cb34c7b4ef19f07522828f66ebf P 7b83c187309b40c8ae16f8149b7ad171: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:17.766973 17180 tablet_bootstrap.cc:492] T 27bf5cb34c7b4ef19f07522828f66ebf P 7b83c187309b40c8ae16f8149b7ad171: No bootstrap required, opened a new log
I20260926 15:03:17.767263 17180 ts_tablet_manager.cc:1397] T 27bf5cb34c7b4ef19f07522828f66ebf P 7b83c187309b40c8ae16f8149b7ad171: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260926 15:03:17.769425 17177 tablet_bootstrap.cc:492] T 27bf5cb34c7b4ef19f07522828f66ebf P b0a766fb9dc74a42967c0463588f3257: No bootstrap required, opened a new log
I20260926 15:03:17.769770 17177 ts_tablet_manager.cc:1397] T 27bf5cb34c7b4ef19f07522828f66ebf P b0a766fb9dc74a42967c0463588f3257: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.000s
I20260926 15:03:17.771665 17176 raft_consensus.cc:348] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedeebde7a6744c69356ec6652b19d3d" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46053 } } peers { permanent_uuid: "b0a766fb9dc74a42967c0463588f3257" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40451 } } peers { permanent_uuid: "7b83c187309b40c8ae16f8149b7ad171" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 43577 } }
I20260926 15:03:17.772118 17176 raft_consensus.cc:374] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:17.772215 17176 raft_consensus.cc:729] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fedeebde7a6744c69356ec6652b19d3d, State: Initialized, Role: FOLLOWER
I20260926 15:03:17.772866 17176 consensus_queue.cc:260] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedeebde7a6744c69356ec6652b19d3d" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46053 } } peers { permanent_uuid: "b0a766fb9dc74a42967c0463588f3257" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40451 } } peers { permanent_uuid: "7b83c187309b40c8ae16f8149b7ad171" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 43577 } }
I20260926 15:03:17.777699 17180 raft_consensus.cc:348] T 27bf5cb34c7b4ef19f07522828f66ebf P 7b83c187309b40c8ae16f8149b7ad171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedeebde7a6744c69356ec6652b19d3d" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46053 } } peers { permanent_uuid: "b0a766fb9dc74a42967c0463588f3257" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40451 } } peers { permanent_uuid: "7b83c187309b40c8ae16f8149b7ad171" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 43577 } }
I20260926 15:03:17.778205 17180 raft_consensus.cc:374] T 27bf5cb34c7b4ef19f07522828f66ebf P 7b83c187309b40c8ae16f8149b7ad171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:17.778319 17180 raft_consensus.cc:729] T 27bf5cb34c7b4ef19f07522828f66ebf P 7b83c187309b40c8ae16f8149b7ad171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b83c187309b40c8ae16f8149b7ad171, State: Initialized, Role: FOLLOWER
I20260926 15:03:17.779112 17180 consensus_queue.cc:260] T 27bf5cb34c7b4ef19f07522828f66ebf P 7b83c187309b40c8ae16f8149b7ad171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedeebde7a6744c69356ec6652b19d3d" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46053 } } peers { permanent_uuid: "b0a766fb9dc74a42967c0463588f3257" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40451 } } peers { permanent_uuid: "7b83c187309b40c8ae16f8149b7ad171" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 43577 } }
I20260926 15:03:17.780854 17177 raft_consensus.cc:348] T 27bf5cb34c7b4ef19f07522828f66ebf P b0a766fb9dc74a42967c0463588f3257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedeebde7a6744c69356ec6652b19d3d" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46053 } } peers { permanent_uuid: "b0a766fb9dc74a42967c0463588f3257" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40451 } } peers { permanent_uuid: "7b83c187309b40c8ae16f8149b7ad171" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 43577 } }
I20260926 15:03:17.781248 17176 ts_tablet_manager.cc:1428] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d: Time spent starting tablet: real 0.018s user 0.016s sys 0.004s
I20260926 15:03:17.781369 17177 raft_consensus.cc:374] T 27bf5cb34c7b4ef19f07522828f66ebf P b0a766fb9dc74a42967c0463588f3257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:17.781476 17177 raft_consensus.cc:729] T 27bf5cb34c7b4ef19f07522828f66ebf P b0a766fb9dc74a42967c0463588f3257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0a766fb9dc74a42967c0463588f3257, State: Initialized, Role: FOLLOWER
I20260926 15:03:17.782164 17177 consensus_queue.cc:260] T 27bf5cb34c7b4ef19f07522828f66ebf P b0a766fb9dc74a42967c0463588f3257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedeebde7a6744c69356ec6652b19d3d" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46053 } } peers { permanent_uuid: "b0a766fb9dc74a42967c0463588f3257" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40451 } } peers { permanent_uuid: "7b83c187309b40c8ae16f8149b7ad171" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 43577 } }
I20260926 15:03:17.789005 17180 ts_tablet_manager.cc:1428] T 27bf5cb34c7b4ef19f07522828f66ebf P 7b83c187309b40c8ae16f8149b7ad171: Time spent starting tablet: real 0.022s user 0.017s sys 0.000s
I20260926 15:03:17.789494 16997 heartbeater.cc:503] Master 127.9.123.126:32823 was elected leader, sending a full tablet report...
I20260926 15:03:17.796922 17139 heartbeater.cc:503] Master 127.9.123.126:32823 was elected leader, sending a full tablet report...
I20260926 15:03:17.797734 17177 ts_tablet_manager.cc:1428] T 27bf5cb34c7b4ef19f07522828f66ebf P b0a766fb9dc74a42967c0463588f3257: Time spent starting tablet: real 0.028s user 0.019s sys 0.009s
I20260926 15:03:17.825712 17182 raft_consensus.cc:482] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:03:17.825996 17182 raft_consensus.cc:504] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedeebde7a6744c69356ec6652b19d3d" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46053 } } peers { permanent_uuid: "b0a766fb9dc74a42967c0463588f3257" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40451 } } peers { permanent_uuid: "7b83c187309b40c8ae16f8149b7ad171" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 43577 } }
I20260926 15:03:17.827579 17182 leader_election.cc:288] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0a766fb9dc74a42967c0463588f3257 (127.9.123.67:40451), 7b83c187309b40c8ae16f8149b7ad171 (127.9.123.66:43577)
I20260926 15:03:17.834292 17173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:17.827858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:58801 (local address 127.9.123.67:40451)
0926 15:03:17.828042 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:17.828049 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:17.829005 (+ 956us) server_negotiation.cc:407] Connection header received
0926 15:03:17.829202 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:17.829207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:17.829276 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:17.829348 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:17.829912 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.830612 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.832510 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.833375 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.833549 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.833635 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:17.833942 (+ 307us) server_negotiation.cc:299] Negotiation successful
0926 15:03:17.834070 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 15:03:17.834327 17175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:17.827946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:57287 (local address 127.9.123.66:43577)
0926 15:03:17.828140 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:17.828147 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:17.829052 (+ 905us) server_negotiation.cc:407] Connection header received
0926 15:03:17.829226 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:17.829230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:17.829284 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:17.829363 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:17.829909 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.830607 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.832523 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.833408 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.833549 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.833634 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:17.833969 (+ 335us) server_negotiation.cc:299] Negotiation successful
0926 15:03:17.834092 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 15:03:17.834461 17185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:17.827768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:40451 (local address 127.9.123.65:58801)
0926 15:03:17.828499 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:17.828514 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:17.829042 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:17.829379 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:17.829383 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:17.829405 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:17.829780 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:17.829788 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.830731 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.830736 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:17.832355 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:17.832369 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.833594 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.833600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:17.833790 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.833821 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 15:03:17.833899 (+ 78us) client_negotiation.cc:240] Negotiation successful
0926 15:03:17.834042 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":43,"thread_start_us":426,"threads_started":1}
I20260926 15:03:17.834573 17186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:17.827831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:43577 (local address 127.9.123.65:57287)
0926 15:03:17.828933 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:17.828948 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:17.829075 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:17.829387 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:17.829390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:17.829430 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:17.829756 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:17.829768 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.830733 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.830738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:17.832332 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:17.832349 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.833633 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.833640 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:17.833827 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.833850 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 15:03:17.833930 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 15:03:17.834060 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"mutex_wait_us":46,"spinlock_wait_cycles":2304,"thread_start_us":411,"threads_started":1}
I20260926 15:03:17.835493 16952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27bf5cb34c7b4ef19f07522828f66ebf" candidate_uuid: "fedeebde7a6744c69356ec6652b19d3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b83c187309b40c8ae16f8149b7ad171" is_pre_election: true
I20260926 15:03:17.835839 17093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27bf5cb34c7b4ef19f07522828f66ebf" candidate_uuid: "fedeebde7a6744c69356ec6652b19d3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0a766fb9dc74a42967c0463588f3257" is_pre_election: true
I20260926 15:03:17.835985 16952 raft_consensus.cc:2454] T 27bf5cb34c7b4ef19f07522828f66ebf P 7b83c187309b40c8ae16f8149b7ad171 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fedeebde7a6744c69356ec6652b19d3d in term 0.
I20260926 15:03:17.836289 17093 raft_consensus.cc:2454] T 27bf5cb34c7b4ef19f07522828f66ebf P b0a766fb9dc74a42967c0463588f3257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fedeebde7a6744c69356ec6652b19d3d in term 0.
I20260926 15:03:17.836602 16740 leader_election.cc:302] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b83c187309b40c8ae16f8149b7ad171, fedeebde7a6744c69356ec6652b19d3d; no voters:
I20260926 15:03:17.837011 17182 raft_consensus.cc:2781] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:03:17.837163 17182 raft_consensus.cc:482] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:03:17.837248 17182 raft_consensus.cc:3037] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:17.839286 17182 raft_consensus.cc:504] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedeebde7a6744c69356ec6652b19d3d" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46053 } } peers { permanent_uuid: "b0a766fb9dc74a42967c0463588f3257" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40451 } } peers { permanent_uuid: "7b83c187309b40c8ae16f8149b7ad171" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 43577 } }
I20260926 15:03:17.839859 17182 leader_election.cc:288] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d [CANDIDATE]: Term 1 election: Requested vote from peers b0a766fb9dc74a42967c0463588f3257 (127.9.123.67:40451), 7b83c187309b40c8ae16f8149b7ad171 (127.9.123.66:43577)
I20260926 15:03:17.840379 16952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27bf5cb34c7b4ef19f07522828f66ebf" candidate_uuid: "fedeebde7a6744c69356ec6652b19d3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b83c187309b40c8ae16f8149b7ad171"
I20260926 15:03:17.840373 17093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27bf5cb34c7b4ef19f07522828f66ebf" candidate_uuid: "fedeebde7a6744c69356ec6652b19d3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0a766fb9dc74a42967c0463588f3257"
I20260926 15:03:17.840577 16952 raft_consensus.cc:3037] T 27bf5cb34c7b4ef19f07522828f66ebf P 7b83c187309b40c8ae16f8149b7ad171 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:17.840577 17093 raft_consensus.cc:3037] T 27bf5cb34c7b4ef19f07522828f66ebf P b0a766fb9dc74a42967c0463588f3257 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:17.843645 16952 raft_consensus.cc:2454] T 27bf5cb34c7b4ef19f07522828f66ebf P 7b83c187309b40c8ae16f8149b7ad171 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fedeebde7a6744c69356ec6652b19d3d in term 1.
I20260926 15:03:17.843654 17093 raft_consensus.cc:2454] T 27bf5cb34c7b4ef19f07522828f66ebf P b0a766fb9dc74a42967c0463588f3257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fedeebde7a6744c69356ec6652b19d3d in term 1.
W20260926 15:03:17.843721 17142 tablet.cc:2369] T 27bf5cb34c7b4ef19f07522828f66ebf P b0a766fb9dc74a42967c0463588f3257: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:03:17.844061 16740 leader_election.cc:302] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b83c187309b40c8ae16f8149b7ad171, fedeebde7a6744c69356ec6652b19d3d; no voters:
I20260926 15:03:17.844323 17182 raft_consensus.cc:2781] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:03:17.844733 17182 raft_consensus.cc:686] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d [term 1 LEADER]: Becoming Leader. State: Replica: fedeebde7a6744c69356ec6652b19d3d, State: Running, Role: LEADER
I20260926 15:03:17.845300 17182 consensus_queue.cc:237] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedeebde7a6744c69356ec6652b19d3d" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46053 } } peers { permanent_uuid: "b0a766fb9dc74a42967c0463588f3257" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40451 } } peers { permanent_uuid: "7b83c187309b40c8ae16f8149b7ad171" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 43577 } }
I20260926 15:03:17.850293 15885 catalog_manager.cc:5495] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d reported cstate change: term changed from 0 to 1, leader changed from <none> to fedeebde7a6744c69356ec6652b19d3d (127.9.123.65). New cstate: current_term: 1 leader_uuid: "fedeebde7a6744c69356ec6652b19d3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedeebde7a6744c69356ec6652b19d3d" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0a766fb9dc74a42967c0463588f3257" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b83c187309b40c8ae16f8149b7ad171" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 43577 } health_report { overall_health: UNKNOWN } } }
I20260926 15:03:17.889664 17171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:17.884797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44126 (local address 127.9.123.65:46053)
0926 15:03:17.884962 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:17.884966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:17.884983 (+ 17us) server_negotiation.cc:407] Connection header received
0926 15:03:17.885162 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:17.885165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:17.885269 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:17.885396 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:03:17.885771 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.886454 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.887739 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.888127 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.888216 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.888603 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:03:17.889175 (+ 572us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:03:17.889280 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:17.889344 (+ 64us) server_negotiation.cc:299] Negotiation successful
0926 15:03:17.889412 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260926 15:03:17.891585 16855 tablet.cc:2369] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:03:17.897943 17093 raft_consensus.cc:1261] T 27bf5cb34c7b4ef19f07522828f66ebf P b0a766fb9dc74a42967c0463588f3257 [term 1 FOLLOWER]: Refusing update from remote peer fedeebde7a6744c69356ec6652b19d3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:03:17.898233 16952 raft_consensus.cc:1261] T 27bf5cb34c7b4ef19f07522828f66ebf P 7b83c187309b40c8ae16f8149b7ad171 [term 1 FOLLOWER]: Refusing update from remote peer fedeebde7a6744c69356ec6652b19d3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:03:17.899127 17187 consensus_queue.cc:1035] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b83c187309b40c8ae16f8149b7ad171" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 43577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:03:17.899502 17182 consensus_queue.cc:1035] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0a766fb9dc74a42967c0463588f3257" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:03:17.910557 17190 mvcc.cc:205] Tried to move back new op lower bound from 7333621751379628032 to 7333621751178055680. Current Snapshot: MvccSnapshot[applied={T|T < 7333621751379628032}]
I20260926 15:03:17.912453 17191 mvcc.cc:205] Tried to move back new op lower bound from 7333621751379628032 to 7333621751178055680. Current Snapshot: MvccSnapshot[applied={T|T < 7333621751379628032}]
I20260926 15:03:17.915139 17192 mvcc.cc:205] Tried to move back new op lower bound from 7333621751379628032 to 7333621751178055680. Current Snapshot: MvccSnapshot[applied={T|T < 7333621751379628032}]
I20260926 15:03:17.938621 15885 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:03:17.938937 15885 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:03:17.945199 15885 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c93ba0d1fefe4605b9d7f761e1e06f96: Sending DeleteTablet for 3 replicas of tablet 27bf5cb34c7b4ef19f07522828f66ebf
I20260926 15:03:17.946419 16932 tablet_service.cc:1514] Processing DeleteTablet for tablet 27bf5cb34c7b4ef19f07522828f66ebf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:03:17 UTC) from {username='oryx', principal='oryx/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:36320
I20260926 15:03:17.946452 17073 tablet_service.cc:1514] Processing DeleteTablet for tablet 27bf5cb34c7b4ef19f07522828f66ebf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:03:17 UTC) from {username='oryx', principal='oryx/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:34850
I20260926 15:03:17.947288 17199 tablet_replica.cc:330] T 27bf5cb34c7b4ef19f07522828f66ebf P 7b83c187309b40c8ae16f8149b7ad171: stopping tablet replica
I20260926 15:03:17.947786 17199 raft_consensus.cc:2229] T 27bf5cb34c7b4ef19f07522828f66ebf P 7b83c187309b40c8ae16f8149b7ad171 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:03:17.948122 16787 tablet_service.cc:1514] Processing DeleteTablet for tablet 27bf5cb34c7b4ef19f07522828f66ebf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:03:17 UTC) from {username='oryx', principal='oryx/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:44122
I20260926 15:03:17.948465 17199 raft_consensus.cc:2258] T 27bf5cb34c7b4ef19f07522828f66ebf P 7b83c187309b40c8ae16f8149b7ad171 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:17.950850 17199 ts_tablet_manager.cc:1905] T 27bf5cb34c7b4ef19f07522828f66ebf P 7b83c187309b40c8ae16f8149b7ad171: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:03:17.958546 17199 ts_tablet_manager.cc:1918] T 27bf5cb34c7b4ef19f07522828f66ebf P 7b83c187309b40c8ae16f8149b7ad171: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:03:17.958846 17199 log.cc:1196] T 27bf5cb34c7b4ef19f07522828f66ebf P 7b83c187309b40c8ae16f8149b7ad171: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/27bf5cb34c7b4ef19f07522828f66ebf
I20260926 15:03:17.959501 17199 ts_tablet_manager.cc:1939] T 27bf5cb34c7b4ef19f07522828f66ebf P 7b83c187309b40c8ae16f8149b7ad171: Deleting consensus metadata
I20260926 15:03:17.961580 15870 catalog_manager.cc:4841] TS 7b83c187309b40c8ae16f8149b7ad171 (127.9.123.66:43577): tablet 27bf5cb34c7b4ef19f07522828f66ebf (table test-table [id=1f32dc15d1a6409e81fdb0d1b9061584]) successfully deleted
I20260926 15:03:17.966596 17201 tablet_replica.cc:330] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d: stopping tablet replica
I20260926 15:03:17.966871 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 15854
I20260926 15:03:17.967969 17201 raft_consensus.cc:2229] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:03:17.968421 15916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:17.971582 17201 raft_consensus.cc:2258] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:17.972657 17200 tablet_replica.cc:330] T 27bf5cb34c7b4ef19f07522828f66ebf P b0a766fb9dc74a42967c0463588f3257: stopping tablet replica
I20260926 15:03:17.973057 17200 raft_consensus.cc:2229] T 27bf5cb34c7b4ef19f07522828f66ebf P b0a766fb9dc74a42967c0463588f3257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:03:17.973603 17200 raft_consensus.cc:2258] T 27bf5cb34c7b4ef19f07522828f66ebf P b0a766fb9dc74a42967c0463588f3257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:17.976470 17200 ts_tablet_manager.cc:1905] T 27bf5cb34c7b4ef19f07522828f66ebf P b0a766fb9dc74a42967c0463588f3257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:03:17.989468 17201 ts_tablet_manager.cc:1905] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:03:18.000859 17200 ts_tablet_manager.cc:1918] T 27bf5cb34c7b4ef19f07522828f66ebf P b0a766fb9dc74a42967c0463588f3257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:03:18.001089 17200 log.cc:1196] T 27bf5cb34c7b4ef19f07522828f66ebf P b0a766fb9dc74a42967c0463588f3257: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/27bf5cb34c7b4ef19f07522828f66ebf
I20260926 15:03:18.001771 17200 ts_tablet_manager.cc:1939] T 27bf5cb34c7b4ef19f07522828f66ebf P b0a766fb9dc74a42967c0463588f3257: Deleting consensus metadata
I20260926 15:03:18.005909 17201 ts_tablet_manager.cc:1918] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:03:18.006098 17201 log.cc:1196] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/wal/wals/27bf5cb34c7b4ef19f07522828f66ebf
I20260926 15:03:18.006112 17143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:17.951086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49562 (local address 127.9.123.125:36901)
0926 15:03:17.951275 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:17.951282 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:17.967269 (+ 15987us) server_negotiation.cc:407] Connection header received
0926 15:03:17.968256 (+ 987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:17.968262 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:17.968363 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:17.968438 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:17.972121 (+ 3683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.973224 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.984296 (+ 11072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:17.985025 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:17.985169 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:17.996556 (+ 11387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:17.996565 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:17.996571 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:17.996630 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:18.000480 (+ 3850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:18.002260 (+ 1780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:18.002267 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:18.002271 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:18.002451 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:18.003053 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:18.003059 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:18.003063 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:18.004046 (+ 983us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:18.004319 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:18.005479 (+ 1160us) server_negotiation.cc:299] Negotiation successful
0926 15:03:18.005755 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 15:03:18.006692 17201 ts_tablet_manager.cc:1939] T 27bf5cb34c7b4ef19f07522828f66ebf P fedeebde7a6744c69356ec6652b19d3d: Deleting consensus metadata
I20260926 15:03:18.022001 17145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:17.966044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47258 (local address 127.9.123.124:45251)
0926 15:03:17.966210 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:17.966217 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:18.008734 (+ 42517us) server_negotiation.cc:407] Connection header received
0926 15:03:18.008890 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:18.008896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:18.008993 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:18.009141 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:18.010868 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:18.011580 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:18.012956 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:18.013501 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:18.013617 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:18.016202 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:18.016208 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:18.016214 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:18.016256 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:18.019416 (+ 3160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:18.020067 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:18.020073 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:18.020077 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:18.020216 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:18.020583 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:18.020588 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:18.020591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:18.020866 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:18.021082 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:18.021461 (+ 379us) server_negotiation.cc:299] Negotiation successful
0926 15:03:18.021676 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 15:03:18.148450 15871 catalog_manager.cc:4841] TS fedeebde7a6744c69356ec6652b19d3d (127.9.123.65:46053): tablet 27bf5cb34c7b4ef19f07522828f66ebf (table test-table [id=1f32dc15d1a6409e81fdb0d1b9061584]) successfully deleted
I20260926 15:03:18.149820 15872 catalog_manager.cc:4841] TS b0a766fb9dc74a42967c0463588f3257 (127.9.123.67:40451): tablet 27bf5cb34c7b4ef19f07522828f66ebf (table test-table [id=1f32dc15d1a6409e81fdb0d1b9061584]) successfully deleted
I20260926 15:03:18.150383 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 15854
I20260926 15:03:18.195989 17143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:18.172632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49576 (local address 127.9.123.125:36901)
0926 15:03:18.172815 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:18.172820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:18.174122 (+ 1302us) server_negotiation.cc:407] Connection header received
0926 15:03:18.174362 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:18.174368 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:18.174443 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:18.174507 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:18.176513 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:18.177680 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:18.179606 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:18.180197 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:18.180299 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:18.189218 (+ 8919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:18.189226 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:18.189231 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:18.189276 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:18.192782 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:18.193630 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:18.193636 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:18.193640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:18.193795 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:18.194196 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:18.194201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:18.194204 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:18.194593 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:18.194844 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:18.195379 (+ 535us) server_negotiation.cc:299] Negotiation successful
0926 15:03:18.195637 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 15:03:18.198184 17145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:18.173056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47272 (local address 127.9.123.124:45251)
0926 15:03:18.173229 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:18.173234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:18.181605 (+ 8371us) server_negotiation.cc:407] Connection header received
0926 15:03:18.181906 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:18.181912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:18.181993 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:18.182059 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:18.184125 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:18.185167 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:18.186976 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:18.187623 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:18.187720 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:18.190898 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:18.190905 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:18.190909 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:18.190954 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:18.194174 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:18.195235 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:18.195241 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:18.195244 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:18.195386 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:18.195834 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:18.195839 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:18.195843 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:18.196184 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:18.196388 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:18.197409 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0926 15:03:18.197638 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 15:03:18.919931 17211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:18.919223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.9.123.65:39167)
0926 15:03:18.919649 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:18.919771 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":215,"threads_started":1}
W20260926 15:03:18.920301 16852 heartbeater.cc:644] Failed to heartbeat to 127.9.123.126:32823 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
I20260926 15:03:18.931010 17212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:18.930275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.9.123.66:45909)
0926 15:03:18.930708 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:18.930837 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":215,"threads_started":1}
W20260926 15:03:18.931624 16997 heartbeater.cc:644] Failed to heartbeat to 127.9.123.126:32823 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
I20260926 15:03:18.932921 17213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:18.931930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.9.123.67:57419)
0926 15:03:18.932538 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:18.932690 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":385,"threads_started":1}
W20260926 15:03:18.933303 17139 heartbeater.cc:644] Failed to heartbeat to 127.9.123.126:32823 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
I20260926 15:03:19.445573 17214 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c93ba0d1fefe4605b9d7f761e1e06f96)
I20260926 15:03:19.445706 17215 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c93ba0d1fefe4605b9d7f761e1e06f96)
I20260926 15:03:19.445816 17214 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: false } }
I20260926 15:03:19.445885 17215 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: false } }
I20260926 15:03:19.446717 17214 leader_election.cc:288] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c93ba0d1fefe4605b9d7f761e1e06f96 (127.9.123.126:32823), 3f2d4d6b2df742a280bec1cc4ef4f615 (127.9.123.125:36901)
I20260926 15:03:19.447861 17217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.446862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.0.0.1:49708)
0926 15:03:19.447392 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.447592 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"threads_started":1}
I20260926 15:03:19.448427 17215 leader_election.cc:288] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c93ba0d1fefe4605b9d7f761e1e06f96 (127.9.123.126:32823), b9febb6a22304d1f9376e29539155b4b (127.9.123.124:45251)
W20260926 15:03:19.448457 16659 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
I20260926 15:03:19.449390 17216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.446862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.0.0.1:49706)
0926 15:03:19.448941 (+ 2079us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.449134 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1954,"thread_start_us":1813,"threads_started":1}
W20260926 15:03:19.449651 16470 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
I20260926 15:03:19.451606 17219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.450945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.0.0.1:49726)
0926 15:03:19.451200 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.451385 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
W20260926 15:03:19.451974 16470 leader_election.cc:334] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c93ba0d1fefe4605b9d7f761e1e06f96 (127.9.123.126:32823): Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
I20260926 15:03:19.452502 17218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.449963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.0.0.1:49720)
0926 15:03:19.452108 (+ 2145us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.452287 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2011}
W20260926 15:03:19.452723 16659 leader_election.cc:334] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c93ba0d1fefe4605b9d7f761e1e06f96 (127.9.123.126:32823): Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
I20260926 15:03:19.457319 17216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.447373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:45251 (local address 127.0.0.1:47284)
0926 15:03:19.449600 (+ 2227us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.449618 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:19.449768 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:19.451495 (+ 1727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:19.451501 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:19.451533 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:19.451778 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:19.451792 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.453628 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.453631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:19.455449 (+ 1818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:19.455463 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.456726 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.456730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:19.456866 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:19.456891 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 15:03:19.456980 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 15:03:19.457079 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2139,"thread_start_us":3529,"threads_started":1}
I20260926 15:03:19.457506 17220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.447801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47284 (local address 127.9.123.124:45251)
0926 15:03:19.451167 (+ 3366us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:19.451173 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:19.451191 (+ 18us) server_negotiation.cc:407] Connection header received
0926 15:03:19.451257 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:19.451262 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:19.451334 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:19.452135 (+ 801us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:19.452381 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.453502 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.455596 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.456568 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.456694 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:19.456786 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:19.457022 (+ 236us) server_negotiation.cc:299] Negotiation successful
0926 15:03:19.457172 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2891,"thread_start_us":2776,"threads_started":1}
I20260926 15:03:19.458074 16684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b9febb6a22304d1f9376e29539155b4b" is_pre_election: true
I20260926 15:03:19.458343 17217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.446940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:36901 (local address 127.0.0.1:49588)
0926 15:03:19.448046 (+ 1106us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.448065 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:19.448218 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:19.452855 (+ 4637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:19.452860 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:19.452885 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:19.453079 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:19.453087 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.454337 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.454342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:19.456057 (+ 1715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:19.456069 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.457564 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.457569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:19.457731 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:19.457763 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 15:03:19.457822 (+ 59us) client_negotiation.cc:240] Negotiation successful
0926 15:03:19.457973 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1010,"thread_start_us":4916,"threads_started":1}
I20260926 15:03:19.458467 16684 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f2d4d6b2df742a280bec1cc4ef4f615 in term 1.
I20260926 15:03:19.458670 17223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.451113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49588 (local address 127.9.123.125:36901)
0926 15:03:19.452602 (+ 1489us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:19.452607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:19.452627 (+ 20us) server_negotiation.cc:407] Connection header received
0926 15:03:19.452677 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:19.452681 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:19.452728 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:19.452817 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:19.453244 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.454175 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.456214 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.457373 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.458139 (+ 766us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:19.458213 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:19.458250 (+ 37us) server_negotiation.cc:299] Negotiation successful
0926 15:03:19.458369 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1386,"thread_start_us":1244,"threads_started":1}
I20260926 15:03:19.458962 16469 leader_election.cc:302] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f2d4d6b2df742a280bec1cc4ef4f615, b9febb6a22304d1f9376e29539155b4b; no voters: c93ba0d1fefe4605b9d7f761e1e06f96
I20260926 15:03:19.459251 17215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260926 15:03:19.459359 17215 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c93ba0d1fefe4605b9d7f761e1e06f96)
I20260926 15:03:19.459458 17215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 1 FOLLOWER]: Advancing to term 2
I20260926 15:03:19.459610 16494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b9febb6a22304d1f9376e29539155b4b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" is_pre_election: true
I20260926 15:03:19.461591 17215 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: false } }
I20260926 15:03:19.461999 16494 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b9febb6a22304d1f9376e29539155b4b in current term 2: Already voted for candidate 3f2d4d6b2df742a280bec1cc4ef4f615 in this term.
I20260926 15:03:19.462234 17215 leader_election.cc:288] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [CANDIDATE]: Term 2 election: Requested vote from peers c93ba0d1fefe4605b9d7f761e1e06f96 (127.9.123.126:32823), b9febb6a22304d1f9376e29539155b4b (127.9.123.124:45251)
I20260926 15:03:19.462688 17216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.462354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.0.0.1:49730)
0926 15:03:19.462498 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.462578 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260926 15:03:19.462663 16684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b9febb6a22304d1f9376e29539155b4b"
I20260926 15:03:19.462888 16684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [term 1 FOLLOWER]: Advancing to term 2
I20260926 15:03:19.463382 16657 leader_election.cc:302] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b9febb6a22304d1f9376e29539155b4b; no voters: 3f2d4d6b2df742a280bec1cc4ef4f615, c93ba0d1fefe4605b9d7f761e1e06f96
I20260926 15:03:19.463622 17216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.463250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.0.0.1:49742)
0926 15:03:19.463424 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.463514 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 15:03:19.463783 16470 leader_election.cc:334] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c93ba0d1fefe4605b9d7f761e1e06f96 (127.9.123.126:32823): Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
I20260926 15:03:19.465301 16684 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f2d4d6b2df742a280bec1cc4ef4f615 in term 2.
I20260926 15:03:19.465507 17214 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 15:03:19.465792 16469 leader_election.cc:302] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f2d4d6b2df742a280bec1cc4ef4f615, b9febb6a22304d1f9376e29539155b4b; no voters: c93ba0d1fefe4605b9d7f761e1e06f96
I20260926 15:03:19.466059 17215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 15:03:19.466564 17215 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [term 2 LEADER]: Becoming Leader. State: Replica: 3f2d4d6b2df742a280bec1cc4ef4f615, State: Running, Role: LEADER
I20260926 15:03:19.466944 17215 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: false } }
I20260926 15:03:19.468847 17224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f2d4d6b2df742a280bec1cc4ef4f615. Latest consensus state: current_term: 2 leader_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: false } } }
I20260926 15:03:19.469216 17224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:19.469861 17226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:03:19.473330 17226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:03:19.474161 17226 catalog_manager.cc:1259] Loaded cluster ID: d728c8a2cf5643cc8a1ea2d38b1a354c
I20260926 15:03:19.474225 17226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:03:19.475242 17226 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:03:19.475976 17226 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615: Loaded TSK: 0
I20260926 15:03:19.476778 17226 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:03:19.490222 16484 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49576:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:03:19.497112 16484 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:03:19.500973 17216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.500616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.0.0.1:49756)
0926 15:03:19.500765 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.500842 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260926 15:03:19.501168 16684 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [term 2 FOLLOWER]: Refusing update from remote peer 3f2d4d6b2df742a280bec1cc4ef4f615: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260926 15:03:19.501884 17216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.501565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.0.0.1:49772)
0926 15:03:19.501692 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.501763 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260926 15:03:19.501876 17224 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260926 15:03:19.502116 16470 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 -> Peer c93ba0d1fefe4605b9d7f761e1e06f96 (127.9.123.126:32823): Couldn't send request to peer c93ba0d1fefe4605b9d7f761e1e06f96. Status: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 15:03:19.504491 17214 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f2d4d6b2df742a280bec1cc4ef4f615. Latest consensus state: current_term: 2 leader_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: false } } }
I20260926 15:03:19.504729 17214 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:03:19.506069 17224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: false } } }
I20260926 15:03:19.506356 17224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:19.507064 17214 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c93ba0d1fefe4605b9d7f761e1e06f96" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 32823 } } peers { permanent_uuid: "3f2d4d6b2df742a280bec1cc4ef4f615" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36901 } attrs { promote: false } } peers { permanent_uuid: "b9febb6a22304d1f9376e29539155b4b" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 45251 } attrs { promote: false } } }
I20260926 15:03:19.507344 17214 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9febb6a22304d1f9376e29539155b4b [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:03:19.522658 17234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.515087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:46053 (local address 127.0.0.1:44128)
0926 15:03:19.515691 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.515705 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:19.515783 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:19.516414 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:19.516419 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:19.516445 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:19.516881 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:19.516895 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.518216 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.518220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:19.519942 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:19.519956 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.521718 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.521724 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:19.521890 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:19.521923 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 15:03:19.522033 (+ 110us) client_negotiation.cc:240] Negotiation successful
0926 15:03:19.522175 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":1,"thread_start_us":320,"threads_started":1}
I20260926 15:03:19.522748 17216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.514209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:40451 (local address 127.0.0.1:34864)
0926 15:03:19.514381 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.514400 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:19.514527 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:19.517129 (+ 2602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:19.517134 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:19.517157 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:19.517359 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:19.517371 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.518901 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.518909 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:19.520569 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:19.520583 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.522088 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.522094 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:19.522264 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:19.522296 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 15:03:19.522369 (+ 73us) client_negotiation.cc:240] Negotiation successful
0926 15:03:19.522491 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":77}
I20260926 15:03:19.523129 17219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.515038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:43577 (local address 127.0.0.1:36334)
0926 15:03:19.515588 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.515625 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:19.515741 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:19.517274 (+ 1533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:19.517278 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:19.517303 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:19.517588 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:19.517598 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.519066 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.519074 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:19.521011 (+ 1937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:19.521023 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.522416 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.522422 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:19.522567 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:19.522592 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 15:03:19.522691 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 15:03:19.522827 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":2}
I20260926 15:03:19.523329 17235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.515322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36334 (local address 127.9.123.66:43577)
0926 15:03:19.516960 (+ 1638us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:19.516966 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:19.516982 (+ 16us) server_negotiation.cc:407] Connection header received
0926 15:03:19.517045 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:19.517050 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:19.517140 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:19.517713 (+ 573us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:19.517906 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.518928 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.521265 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.522147 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.522344 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:19.522442 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:19.522720 (+ 278us) server_negotiation.cc:299] Negotiation successful
0926 15:03:19.523015 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1529,"thread_start_us":2303,"threads_started":1}
I20260926 15:03:19.523545 17233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.514424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34864 (local address 127.9.123.67:40451)
0926 15:03:19.516791 (+ 2367us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:19.516797 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:19.516815 (+ 18us) server_negotiation.cc:407] Connection header received
0926 15:03:19.516878 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:19.516883 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:19.516959 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:19.517502 (+ 543us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:19.517678 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.518743 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.520765 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.521939 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.522954 (+ 1015us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:19.523063 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:19.523114 (+ 51us) server_negotiation.cc:299] Negotiation successful
0926 15:03:19.523252 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2254,"thread_start_us":2161,"threads_started":1}
I20260926 15:03:19.524314 16932 tablet_service.cc:1467] Processing CreateTablet for tablet bba43ae76cbc478da7d9b785ef16f23e (DEFAULT_TABLE table=test-table [id=7dbcd23e4f96446b8e82e91941bb3288]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:03:19.524535 17073 tablet_service.cc:1467] Processing CreateTablet for tablet bba43ae76cbc478da7d9b785ef16f23e (DEFAULT_TABLE table=test-table [id=7dbcd23e4f96446b8e82e91941bb3288]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:03:19.524812 16932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bba43ae76cbc478da7d9b785ef16f23e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:19.524971 17073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bba43ae76cbc478da7d9b785ef16f23e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:19.527933 17236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.515466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44128 (local address 127.9.123.65:46053)
0926 15:03:19.516029 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:19.516035 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:19.516055 (+ 20us) server_negotiation.cc:407] Connection header received
0926 15:03:19.516128 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:19.516133 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:19.516220 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:19.516341 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:19.517144 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.518069 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.520108 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.521346 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.521467 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:19.521582 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:19.527440 (+ 5858us) server_negotiation.cc:299] Negotiation successful
0926 15:03:19.527625 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":334,"threads_started":1}
I20260926 15:03:19.528960 16787 tablet_service.cc:1467] Processing CreateTablet for tablet bba43ae76cbc478da7d9b785ef16f23e (DEFAULT_TABLE table=test-table [id=7dbcd23e4f96446b8e82e91941bb3288]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:03:19.529433 16787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bba43ae76cbc478da7d9b785ef16f23e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:19.537372 17239 tablet_bootstrap.cc:492] T bba43ae76cbc478da7d9b785ef16f23e P fedeebde7a6744c69356ec6652b19d3d: Bootstrap starting.
I20260926 15:03:19.538318 17237 tablet_bootstrap.cc:492] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171: Bootstrap starting.
I20260926 15:03:19.539383 17239 tablet_bootstrap.cc:654] T bba43ae76cbc478da7d9b785ef16f23e P fedeebde7a6744c69356ec6652b19d3d: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:19.540257 17237 tablet_bootstrap.cc:654] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:19.542201 17239 tablet_bootstrap.cc:492] T bba43ae76cbc478da7d9b785ef16f23e P fedeebde7a6744c69356ec6652b19d3d: No bootstrap required, opened a new log
I20260926 15:03:19.542351 17239 ts_tablet_manager.cc:1397] T bba43ae76cbc478da7d9b785ef16f23e P fedeebde7a6744c69356ec6652b19d3d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260926 15:03:19.542963 17239 raft_consensus.cc:348] T bba43ae76cbc478da7d9b785ef16f23e P fedeebde7a6744c69356ec6652b19d3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0a766fb9dc74a42967c0463588f3257" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40451 } } peers { permanent_uuid: "fedeebde7a6744c69356ec6652b19d3d" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46053 } } peers { permanent_uuid: "7b83c187309b40c8ae16f8149b7ad171" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 43577 } }
I20260926 15:03:19.543147 17239 raft_consensus.cc:374] T bba43ae76cbc478da7d9b785ef16f23e P fedeebde7a6744c69356ec6652b19d3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:19.543241 17239 raft_consensus.cc:729] T bba43ae76cbc478da7d9b785ef16f23e P fedeebde7a6744c69356ec6652b19d3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fedeebde7a6744c69356ec6652b19d3d, State: Initialized, Role: FOLLOWER
I20260926 15:03:19.543450 17239 consensus_queue.cc:260] T bba43ae76cbc478da7d9b785ef16f23e P fedeebde7a6744c69356ec6652b19d3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0a766fb9dc74a42967c0463588f3257" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40451 } } peers { permanent_uuid: "fedeebde7a6744c69356ec6652b19d3d" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46053 } } peers { permanent_uuid: "7b83c187309b40c8ae16f8149b7ad171" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 43577 } }
I20260926 15:03:19.543852 17238 tablet_bootstrap.cc:492] T bba43ae76cbc478da7d9b785ef16f23e P b0a766fb9dc74a42967c0463588f3257: Bootstrap starting.
I20260926 15:03:19.545652 17238 tablet_bootstrap.cc:654] T bba43ae76cbc478da7d9b785ef16f23e P b0a766fb9dc74a42967c0463588f3257: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:19.546933 17239 ts_tablet_manager.cc:1428] T bba43ae76cbc478da7d9b785ef16f23e P fedeebde7a6744c69356ec6652b19d3d: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260926 15:03:19.548990 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.544540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.9.123.65:60359)
0926 15:03:19.548691 (+ 4151us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.548805 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4027,"thread_start_us":3902,"threads_started":1}
I20260926 15:03:19.549320 17237 tablet_bootstrap.cc:492] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171: No bootstrap required, opened a new log
I20260926 15:03:19.549485 17237 ts_tablet_manager.cc:1397] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.005s
I20260926 15:03:19.550278 17237 raft_consensus.cc:348] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0a766fb9dc74a42967c0463588f3257" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40451 } } peers { permanent_uuid: "fedeebde7a6744c69356ec6652b19d3d" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46053 } } peers { permanent_uuid: "7b83c187309b40c8ae16f8149b7ad171" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 43577 } }
I20260926 15:03:19.550516 17237 raft_consensus.cc:374] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:19.550611 17237 raft_consensus.cc:729] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b83c187309b40c8ae16f8149b7ad171, State: Initialized, Role: FOLLOWER
I20260926 15:03:19.550870 17245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.549561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.9.123.65:34957)
0926 15:03:19.550327 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.550461 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":517,"threads_started":1}
I20260926 15:03:19.550935 16854 heartbeater.cc:503] Master 127.9.123.125:36901 was elected leader, sending a full tablet report...
I20260926 15:03:19.550868 17237 consensus_queue.cc:260] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0a766fb9dc74a42967c0463588f3257" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40451 } } peers { permanent_uuid: "fedeebde7a6744c69356ec6652b19d3d" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46053 } } peers { permanent_uuid: "7b83c187309b40c8ae16f8149b7ad171" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 43577 } }
I20260926 15:03:19.551668 17237 ts_tablet_manager.cc:1428] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 15:03:19.552142 17238 tablet_bootstrap.cc:492] T bba43ae76cbc478da7d9b785ef16f23e P b0a766fb9dc74a42967c0463588f3257: No bootstrap required, opened a new log
I20260926 15:03:19.552505 17238 ts_tablet_manager.cc:1397] T bba43ae76cbc478da7d9b785ef16f23e P b0a766fb9dc74a42967c0463588f3257: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260926 15:03:19.553608 17238 raft_consensus.cc:348] T bba43ae76cbc478da7d9b785ef16f23e P b0a766fb9dc74a42967c0463588f3257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0a766fb9dc74a42967c0463588f3257" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40451 } } peers { permanent_uuid: "fedeebde7a6744c69356ec6652b19d3d" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46053 } } peers { permanent_uuid: "7b83c187309b40c8ae16f8149b7ad171" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 43577 } }
I20260926 15:03:19.553833 17238 raft_consensus.cc:374] T bba43ae76cbc478da7d9b785ef16f23e P b0a766fb9dc74a42967c0463588f3257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:19.553922 17238 raft_consensus.cc:729] T bba43ae76cbc478da7d9b785ef16f23e P b0a766fb9dc74a42967c0463588f3257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0a766fb9dc74a42967c0463588f3257, State: Initialized, Role: FOLLOWER
I20260926 15:03:19.554152 17238 consensus_queue.cc:260] T bba43ae76cbc478da7d9b785ef16f23e P b0a766fb9dc74a42967c0463588f3257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0a766fb9dc74a42967c0463588f3257" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40451 } } peers { permanent_uuid: "fedeebde7a6744c69356ec6652b19d3d" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46053 } } peers { permanent_uuid: "7b83c187309b40c8ae16f8149b7ad171" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 43577 } }
I20260926 15:03:19.554431 17247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.551884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.9.123.66:52439)
0926 15:03:19.554010 (+ 2126us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.554156 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2010,"thread_start_us":2262,"threads_started":1}
I20260926 15:03:19.555299 17238 ts_tablet_manager.cc:1428] T bba43ae76cbc478da7d9b785ef16f23e P b0a766fb9dc74a42967c0463588f3257: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260926 15:03:19.555509 16999 heartbeater.cc:503] Master 127.9.123.125:36901 was elected leader, sending a full tablet report...
I20260926 15:03:19.555934 17247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.555017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.9.123.66:44839)
0926 15:03:19.555136 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.555219 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260926 15:03:19.559032 17249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.555797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.9.123.67:46129)
0926 15:03:19.558656 (+ 2859us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.558798 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2148,"thread_start_us":2001,"threads_started":1}
I20260926 15:03:19.560042 17249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.559697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.9.123.67:49023)
0926 15:03:19.559867 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.559938 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260926 15:03:19.711767 17246 raft_consensus.cc:482] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:03:19.711962 17246 raft_consensus.cc:504] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0a766fb9dc74a42967c0463588f3257" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40451 } } peers { permanent_uuid: "fedeebde7a6744c69356ec6652b19d3d" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46053 } } peers { permanent_uuid: "7b83c187309b40c8ae16f8149b7ad171" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 43577 } }
I20260926 15:03:19.713555 17246 leader_election.cc:288] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0a766fb9dc74a42967c0463588f3257 (127.9.123.67:40451), fedeebde7a6744c69356ec6652b19d3d (127.9.123.65:46053)
I20260926 15:03:19.720947 17233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.713942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:37123 (local address 127.9.123.67:40451)
0926 15:03:19.714118 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:19.714124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:19.714146 (+ 22us) server_negotiation.cc:407] Connection header received
0926 15:03:19.714255 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:19.714262 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:19.714342 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:19.714428 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:19.714958 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.716036 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.718607 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.719852 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.720030 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:19.720108 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:19.720559 (+ 451us) server_negotiation.cc:299] Negotiation successful
0926 15:03:19.720704 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 15:03:19.720954 17236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.713878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:40831 (local address 127.9.123.65:46053)
0926 15:03:19.714048 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:19.714054 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:19.714699 (+ 645us) server_negotiation.cc:407] Connection header received
0926 15:03:19.714834 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:19.714838 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:19.714895 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:19.714983 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:19.715369 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.716297 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.718607 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.719852 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.720031 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:19.720108 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:19.720559 (+ 451us) server_negotiation.cc:299] Negotiation successful
0926 15:03:19.720696 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 15:03:19.721000 17247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.713764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:40451 (local address 127.9.123.66:37123)
0926 15:03:19.713949 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.713969 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:19.714116 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:19.714503 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:19.714507 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:19.714532 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:19.714826 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:19.714837 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.716188 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.716192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:19.718414 (+ 2222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:19.718434 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.720100 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.720106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:19.720356 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:19.720418 (+ 62us) client_negotiation.cc:769] Sending connection context
0926 15:03:19.720514 (+ 96us) client_negotiation.cc:240] Negotiation successful
0926 15:03:19.720674 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":86}
I20260926 15:03:19.721060 17250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.713778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:46053 (local address 127.9.123.66:40831)
0926 15:03:19.714591 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.714612 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:19.714726 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:19.714992 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:19.714995 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:19.715013 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:19.715255 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:19.715262 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.716404 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.716409 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:19.718437 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:19.718446 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.720099 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.720106 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:19.720351 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:19.720423 (+ 72us) client_negotiation.cc:769] Sending connection context
0926 15:03:19.720514 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 15:03:19.720674 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"mutex_wait_us":333,"thread_start_us":269,"threads_started":1}
I20260926 15:03:19.722178 17093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bba43ae76cbc478da7d9b785ef16f23e" candidate_uuid: "7b83c187309b40c8ae16f8149b7ad171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0a766fb9dc74a42967c0463588f3257" is_pre_election: true
I20260926 15:03:19.722167 16807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bba43ae76cbc478da7d9b785ef16f23e" candidate_uuid: "7b83c187309b40c8ae16f8149b7ad171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fedeebde7a6744c69356ec6652b19d3d" is_pre_election: true
I20260926 15:03:19.722432 17093 raft_consensus.cc:2454] T bba43ae76cbc478da7d9b785ef16f23e P b0a766fb9dc74a42967c0463588f3257 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b83c187309b40c8ae16f8149b7ad171 in term 0.
I20260926 15:03:19.722504 16807 raft_consensus.cc:2454] T bba43ae76cbc478da7d9b785ef16f23e P fedeebde7a6744c69356ec6652b19d3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b83c187309b40c8ae16f8149b7ad171 in term 0.
I20260926 15:03:19.722954 16887 leader_election.cc:302] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7b83c187309b40c8ae16f8149b7ad171, b0a766fb9dc74a42967c0463588f3257, fedeebde7a6744c69356ec6652b19d3d; no voters:
I20260926 15:03:19.723444 17246 raft_consensus.cc:2781] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:03:19.723562 17246 raft_consensus.cc:482] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:03:19.723645 17246 raft_consensus.cc:3037] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:19.725754 17246 raft_consensus.cc:504] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0a766fb9dc74a42967c0463588f3257" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40451 } } peers { permanent_uuid: "fedeebde7a6744c69356ec6652b19d3d" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46053 } } peers { permanent_uuid: "7b83c187309b40c8ae16f8149b7ad171" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 43577 } }
I20260926 15:03:19.726325 17246 leader_election.cc:288] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171 [CANDIDATE]: Term 1 election: Requested vote from peers b0a766fb9dc74a42967c0463588f3257 (127.9.123.67:40451), fedeebde7a6744c69356ec6652b19d3d (127.9.123.65:46053)
I20260926 15:03:19.726763 17093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bba43ae76cbc478da7d9b785ef16f23e" candidate_uuid: "7b83c187309b40c8ae16f8149b7ad171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0a766fb9dc74a42967c0463588f3257"
I20260926 15:03:19.726765 16807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bba43ae76cbc478da7d9b785ef16f23e" candidate_uuid: "7b83c187309b40c8ae16f8149b7ad171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fedeebde7a6744c69356ec6652b19d3d"
I20260926 15:03:19.726933 16807 raft_consensus.cc:3037] T bba43ae76cbc478da7d9b785ef16f23e P fedeebde7a6744c69356ec6652b19d3d [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:19.726938 17093 raft_consensus.cc:3037] T bba43ae76cbc478da7d9b785ef16f23e P b0a766fb9dc74a42967c0463588f3257 [term 0 FOLLOWER]: Advancing to term 1
W20260926 15:03:19.728590 17000 tablet.cc:2369] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:03:19.729022 16807 raft_consensus.cc:2454] T bba43ae76cbc478da7d9b785ef16f23e P fedeebde7a6744c69356ec6652b19d3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b83c187309b40c8ae16f8149b7ad171 in term 1.
I20260926 15:03:19.729429 16886 leader_election.cc:302] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b83c187309b40c8ae16f8149b7ad171, fedeebde7a6744c69356ec6652b19d3d; no voters:
I20260926 15:03:19.729460 17093 raft_consensus.cc:2454] T bba43ae76cbc478da7d9b785ef16f23e P b0a766fb9dc74a42967c0463588f3257 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b83c187309b40c8ae16f8149b7ad171 in term 1.
I20260926 15:03:19.729696 17246 raft_consensus.cc:2781] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:03:19.730136 17246 raft_consensus.cc:686] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171 [term 1 LEADER]: Becoming Leader. State: Replica: 7b83c187309b40c8ae16f8149b7ad171, State: Running, Role: LEADER
I20260926 15:03:19.730595 17246 consensus_queue.cc:237] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0a766fb9dc74a42967c0463588f3257" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40451 } } peers { permanent_uuid: "fedeebde7a6744c69356ec6652b19d3d" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46053 } } peers { permanent_uuid: "7b83c187309b40c8ae16f8149b7ad171" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 43577 } }
I20260926 15:03:19.731055 17247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.730656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.9.123.66:47121)
0926 15:03:19.730825 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.730933 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260926 15:03:19.734112 16483 catalog_manager.cc:5495] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b83c187309b40c8ae16f8149b7ad171 (127.9.123.66). New cstate: current_term: 1 leader_uuid: "7b83c187309b40c8ae16f8149b7ad171" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0a766fb9dc74a42967c0463588f3257" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fedeebde7a6744c69356ec6652b19d3d" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b83c187309b40c8ae16f8149b7ad171" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 43577 } health_report { overall_health: HEALTHY } } }
I20260926 15:03:19.767205 17235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.762026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54516 (local address 127.9.123.66:43577)
0926 15:03:19.762248 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:19.762253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:19.762273 (+ 20us) server_negotiation.cc:407] Connection header received
0926 15:03:19.762500 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:19.762504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:19.762592 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:19.762727 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:03:19.763104 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.763815 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.765144 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.765554 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.765663 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:19.766087 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:03:19.766624 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:03:19.766748 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:19.766858 (+ 110us) server_negotiation.cc:299] Negotiation successful
0926 15:03:19.766941 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 15:03:19.771222 17093 raft_consensus.cc:1261] T bba43ae76cbc478da7d9b785ef16f23e P b0a766fb9dc74a42967c0463588f3257 [term 1 FOLLOWER]: Refusing update from remote peer 7b83c187309b40c8ae16f8149b7ad171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:03:19.771548 16807 raft_consensus.cc:1261] T bba43ae76cbc478da7d9b785ef16f23e P fedeebde7a6744c69356ec6652b19d3d [term 1 FOLLOWER]: Refusing update from remote peer 7b83c187309b40c8ae16f8149b7ad171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:03:19.771886 17251 consensus_queue.cc:1035] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0a766fb9dc74a42967c0463588f3257" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:03:19.772233 17246 consensus_queue.cc:1035] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fedeebde7a6744c69356ec6652b19d3d" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:03:19.773576 17141 heartbeater.cc:503] Master 127.9.123.125:36901 was elected leader, sending a full tablet report...
I20260926 15:03:19.774382 17249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.773900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.9.123.67:38227)
0926 15:03:19.774038 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.774155 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260926 15:03:19.774932 17245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.774576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.9.123.65:55833)
0926 15:03:19.774706 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.774792 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260926 15:03:19.780318 17249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.779773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.9.123.67:59355)
0926 15:03:19.780013 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.780110 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260926 15:03:19.788309 17247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.787781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.9.123.66:52029)
0926 15:03:19.787983 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.788078 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260926 15:03:19.792598 17247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.792222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.9.123.66:47355)
0926 15:03:19.792365 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.792442 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260926 15:03:19.794341 17245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.793880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.9.123.65:32999)
0926 15:03:19.794120 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.794199 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260926 15:03:19.801362 16483 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:03:19.801668 16483 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:03:19.807459 16483 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f2d4d6b2df742a280bec1cc4ef4f615: Sending DeleteTablet for 3 replicas of tablet bba43ae76cbc478da7d9b785ef16f23e
I20260926 15:03:19.808472 16787 tablet_service.cc:1514] Processing DeleteTablet for tablet bba43ae76cbc478da7d9b785ef16f23e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:03:19 UTC) from {username='oryx', principal='oryx/127.9.123.125@KRBTEST.COM'} at 127.0.0.1:44128
I20260926 15:03:19.808454 17073 tablet_service.cc:1514] Processing DeleteTablet for tablet bba43ae76cbc478da7d9b785ef16f23e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:03:19 UTC) from {username='oryx', principal='oryx/127.9.123.125@KRBTEST.COM'} at 127.0.0.1:34864
I20260926 15:03:19.808812 16932 tablet_service.cc:1514] Processing DeleteTablet for tablet bba43ae76cbc478da7d9b785ef16f23e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:03:19 UTC) from {username='oryx', principal='oryx/127.9.123.125@KRBTEST.COM'} at 127.0.0.1:36334
I20260926 15:03:19.811115 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 16679
I20260926 15:03:19.821619 17265 tablet_replica.cc:330] T bba43ae76cbc478da7d9b785ef16f23e P fedeebde7a6744c69356ec6652b19d3d: stopping tablet replica
I20260926 15:03:19.822013 17265 raft_consensus.cc:2229] T bba43ae76cbc478da7d9b785ef16f23e P fedeebde7a6744c69356ec6652b19d3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:03:19.822474 17265 raft_consensus.cc:2258] T bba43ae76cbc478da7d9b785ef16f23e P fedeebde7a6744c69356ec6652b19d3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:19.823786 17265 ts_tablet_manager.cc:1905] T bba43ae76cbc478da7d9b785ef16f23e P fedeebde7a6744c69356ec6652b19d3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:03:19.825523 17264 tablet_replica.cc:330] T bba43ae76cbc478da7d9b785ef16f23e P b0a766fb9dc74a42967c0463588f3257: stopping tablet replica
I20260926 15:03:19.825675 17266 tablet_replica.cc:330] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171: stopping tablet replica
I20260926 15:03:19.825701 17264 raft_consensus.cc:2229] T bba43ae76cbc478da7d9b785ef16f23e P b0a766fb9dc74a42967c0463588f3257 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:03:19.825891 17266 raft_consensus.cc:2229] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171 [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:03:19.825982 17264 raft_consensus.cc:2258] T bba43ae76cbc478da7d9b785ef16f23e P b0a766fb9dc74a42967c0463588f3257 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:19.826490 17266 raft_consensus.cc:2258] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:19.830173 17266 ts_tablet_manager.cc:1905] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:03:19.830821 17265 ts_tablet_manager.cc:1918] T bba43ae76cbc478da7d9b785ef16f23e P fedeebde7a6744c69356ec6652b19d3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:03:19.830955 17265 log.cc:1196] T bba43ae76cbc478da7d9b785ef16f23e P fedeebde7a6744c69356ec6652b19d3d: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-0/wal/wals/bba43ae76cbc478da7d9b785ef16f23e
I20260926 15:03:19.831310 17265 ts_tablet_manager.cc:1939] T bba43ae76cbc478da7d9b785ef16f23e P fedeebde7a6744c69356ec6652b19d3d: Deleting consensus metadata
I20260926 15:03:19.832397 16469 catalog_manager.cc:4841] TS fedeebde7a6744c69356ec6652b19d3d (127.9.123.65:46053): tablet bba43ae76cbc478da7d9b785ef16f23e (table test-table [id=7dbcd23e4f96446b8e82e91941bb3288]) successfully deleted
I20260926 15:03:19.832995 17264 ts_tablet_manager.cc:1905] T bba43ae76cbc478da7d9b785ef16f23e P b0a766fb9dc74a42967c0463588f3257: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:03:19.839193 17266 ts_tablet_manager.cc:1918] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:03:19.839195 17264 ts_tablet_manager.cc:1918] T bba43ae76cbc478da7d9b785ef16f23e P b0a766fb9dc74a42967c0463588f3257: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:03:19.839339 17264 log.cc:1196] T bba43ae76cbc478da7d9b785ef16f23e P b0a766fb9dc74a42967c0463588f3257: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/bba43ae76cbc478da7d9b785ef16f23e
I20260926 15:03:19.839344 17266 log.cc:1196] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/bba43ae76cbc478da7d9b785ef16f23e
I20260926 15:03:19.839774 17264 ts_tablet_manager.cc:1939] T bba43ae76cbc478da7d9b785ef16f23e P b0a766fb9dc74a42967c0463588f3257: Deleting consensus metadata
I20260926 15:03:19.839854 17266 ts_tablet_manager.cc:1939] T bba43ae76cbc478da7d9b785ef16f23e P 7b83c187309b40c8ae16f8149b7ad171: Deleting consensus metadata
I20260926 15:03:19.840960 16470 catalog_manager.cc:4841] TS b0a766fb9dc74a42967c0463588f3257 (127.9.123.67:40451): tablet bba43ae76cbc478da7d9b785ef16f23e (table test-table [id=7dbcd23e4f96446b8e82e91941bb3288]) successfully deleted
I20260926 15:03:19.840989 16468 catalog_manager.cc:4841] TS 7b83c187309b40c8ae16f8149b7ad171 (127.9.123.66:43577): tablet bba43ae76cbc478da7d9b785ef16f23e (table test-table [id=7dbcd23e4f96446b8e82e91941bb3288]) successfully deleted
I20260926 15:03:19.845037 17236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.811850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35650 (local address 127.9.123.65:46053)
0926 15:03:19.812043 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:19.812049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:19.827197 (+ 15148us) server_negotiation.cc:407] Connection header received
0926 15:03:19.827387 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:19.827393 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:19.827470 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:19.827572 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:19.829573 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.830654 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.834071 (+ 3417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.834672 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.834795 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:19.838228 (+ 3433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:19.838247 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:19.838267 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:19.838314 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:19.842052 (+ 3738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:19.842729 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:19.842749 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:19.842753 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:19.842924 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:19.843337 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:19.843342 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:19.843345 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:19.843833 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:19.844121 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:19.844453 (+ 332us) server_negotiation.cc:299] Negotiation successful
0926 15:03:19.844673 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 15:03:19.845752 16847 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:19.896567 17234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.896091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.0.0.1:33830)
0926 15:03:19.896279 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.896397 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 15:03:19.897728 17234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.897320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:32823 (local address 127.0.0.1:33834)
0926 15:03:19.897475 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:19.897564 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260926 15:03:19.961059 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 16679
I20260926 15:03:19.982239 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 16864
I20260926 15:03:19.999814 17235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:19.982997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54518 (local address 127.9.123.66:43577)
0926 15:03:19.983156 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:19.983162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:19.983231 (+ 69us) server_negotiation.cc:407] Connection header received
0926 15:03:19.983435 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:19.983440 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:19.983513 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:19.983618 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:19.985463 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.986688 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.988522 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:19.989382 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:19.989517 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:19.992766 (+ 3249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:19.992783 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:19.992804 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:19.992851 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:19.997012 (+ 4161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:19.997784 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:19.997794 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:19.997798 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:19.997975 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:19.998297 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:19.998302 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:19.998306 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:19.998756 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:19.999052 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:19.999233 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 15:03:19.999446 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 15:03:20.000473 16992 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:20.115141 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 16864
I20260926 15:03:20.130390 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 17010
I20260926 15:03:20.143380 17233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:20.131117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56102 (local address 127.9.123.67:40451)
0926 15:03:20.131283 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:20.131289 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:20.131343 (+ 54us) server_negotiation.cc:407] Connection header received
0926 15:03:20.131528 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:20.131533 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:20.131593 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:20.131672 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:20.133165 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:20.133883 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:20.135072 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:20.135478 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:20.135583 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:20.138015 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:20.138030 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:20.138046 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:20.138084 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:20.140706 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:20.141475 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:20.141482 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:20.141485 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:20.141612 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:20.141943 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:20.141947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:20.141949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:20.142290 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:20.142567 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:20.142870 (+ 303us) server_negotiation.cc:299] Negotiation successful
0926 15:03:20.143088 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 15:03:20.143988 17134 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:20.237332 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 17010
I20260926 15:03:20.252496 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 16347
I20260926 15:03:20.253713 16514 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:20.371675 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 16347
I20260926 15:03:20.388361 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 16447
I20260926 15:03:20.389421 16704 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:20.501762 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 16447
2026-09-26T15:03:20Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9383 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:03:20 dist-test-slave-7153 krb5kdc[17275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:03:20 dist-test-slave-7153 krb5kdc[17275](info): set up 2 sockets
Sep 26 15:03:20 dist-test-slave-7153 krb5kdc[17275](info): commencing operation
krb5kdc: starting...
W20260926 15:03:21.521870 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.979s user 0.001s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:03:21 dist-test-slave-7153 krb5kdc[17275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:03:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:03:21Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.126.keytab.
Entry for principal oryx/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.126.keytab.
I20260926 15:03:21.732488 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:41723
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.126.keytab
--principal=oryx/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:36137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:41723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:03:22.054943 17288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:22.055238 17288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:22.055325 17288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:22.062613 17288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:03:22.062731 17288 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260926 15:03:22.062770 17288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:22.062827 17288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:22.062891 17288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:03:22.062952 17288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:03:22.072927 17288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:36137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:41723
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.126.keytab
--principal=oryx/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:41723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:22.074255 17288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:22.075583 17288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:22.087270 17296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:22.087225 17294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:22.087292 17293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:22.088368 17288 server_base.cc:1034] running on GCE node
I20260926 15:03:22.089231 17288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:22.090682 17288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:22.091909 17288 hybrid_clock.cc:648] HybridClock initialized: now 1790435002091844 us; error 45 us; skew 500 ppm
I20260926 15:03:22.092228 17288 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:22 dist-test-slave-7153 krb5kdc[17275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435002, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:22.095342 17288 init.cc:377] Logged in from keytab as oryx/127.9.123.126@KRBTEST.COM (short username oryx)
I20260926 15:03:22.096903 17288 webserver.cc:460] Webserver started at http://127.9.123.126:37893/ using document root <none> and password file <none>
I20260926 15:03:22.097659 17288 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:22.097764 17288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:22.098085 17288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:22.101162 17288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "f6315f47bfa9454faf8479114234aa07"
format_stamp: "Formatted at 2026-09-26 15:03:22 on dist-test-slave-7153"
server_key: "952466fd58dca0a4ffe93953469e9ceb"
server_key_iv: "942b8434b8f03502a3c74c8dd9f40932"
server_key_version: "encryptionkey@0"
I20260926 15:03:22.102041 17288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "f6315f47bfa9454faf8479114234aa07"
format_stamp: "Formatted at 2026-09-26 15:03:22 on dist-test-slave-7153"
server_key: "952466fd58dca0a4ffe93953469e9ceb"
server_key_iv: "942b8434b8f03502a3c74c8dd9f40932"
server_key_version: "encryptionkey@0"
I20260926 15:03:22.107487 17288 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 15:03:22.111343 17303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:22.112885 17288 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 15:03:22.113049 17288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "f6315f47bfa9454faf8479114234aa07"
format_stamp: "Formatted at 2026-09-26 15:03:22 on dist-test-slave-7153"
server_key: "952466fd58dca0a4ffe93953469e9ceb"
server_key_iv: "942b8434b8f03502a3c74c8dd9f40932"
server_key_version: "encryptionkey@0"
I20260926 15:03:22.113294 17288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:22.127427 17288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:22.131982 17288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:22.132243 17288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:22.152757 17288 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:41723
I20260926 15:03:22.152768 17364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:41723 every 8 connection(s)
I20260926 15:03:22.154145 17288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:03:22.158232 17365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:22.162662 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 17288
I20260926 15:03:22.162859 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:03:22.163478 9709 external_mini_cluster.cc:1428] Setting key bf0e4cd772f68a8ed5c313796cb4b6c1
I20260926 15:03:22.169901 17365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07: Bootstrap starting.
I20260926 15:03:22.173580 17365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:22.174736 17365 log.cc:824] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:22.178143 17365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07: No bootstrap required, opened a new log
Sep 26 15:03:22 dist-test-slave-7153 krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM
I20260926 15:03:22.185729 17365 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6315f47bfa9454faf8479114234aa07" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 41723 } }
I20260926 15:03:22.186112 17365 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:22.186249 17365 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6315f47bfa9454faf8479114234aa07, State: Initialized, Role: FOLLOWER
I20260926 15:03:22.186595 17368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:22.165633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37570 (local address 127.9.123.126:41723)
0926 15:03:22.166412 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:22.166424 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:22.166465 (+ 41us) server_negotiation.cc:407] Connection header received
0926 15:03:22.168317 (+ 1852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:22.168349 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:22.168919 (+ 570us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:22.169360 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:22.171044 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:22.172389 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:22.174111 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:22.174812 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:22.174988 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:22.179490 (+ 4502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:22.179499 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:22.179513 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:22.179559 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:22.182531 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:22.183196 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:22.183203 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:22.183213 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:22.183385 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:22.183761 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:22.183765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:22.183767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:22.184342 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:22.184624 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:22.185051 (+ 427us) server_negotiation.cc:299] Negotiation successful
0926 15:03:22.185466 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":344,"threads_started":1}
I20260926 15:03:22.186844 17365 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6315f47bfa9454faf8479114234aa07" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 41723 } }
I20260926 15:03:22.187079 17365 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:03:22.187227 17365 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:03:22.187388 17365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:22.189757 17365 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6315f47bfa9454faf8479114234aa07" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 41723 } }
I20260926 15:03:22.190227 17365 leader_election.cc:302] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6315f47bfa9454faf8479114234aa07; no voters:
I20260926 15:03:22.190785 17365 leader_election.cc:288] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:03:22.190862 17370 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:03:22.192011 17370 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07 [term 1 LEADER]: Becoming Leader. State: Replica: f6315f47bfa9454faf8479114234aa07, State: Running, Role: LEADER
I20260926 15:03:22.192494 17370 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6315f47bfa9454faf8479114234aa07" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 41723 } }
I20260926 15:03:22.192811 17365 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:03:22.195358 17371 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6315f47bfa9454faf8479114234aa07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6315f47bfa9454faf8479114234aa07" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 41723 } } }
I20260926 15:03:22.195384 17372 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6315f47bfa9454faf8479114234aa07. Latest consensus state: current_term: 1 leader_uuid: "f6315f47bfa9454faf8479114234aa07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6315f47bfa9454faf8479114234aa07" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 41723 } } }
I20260926 15:03:22.195845 17371 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:22.195873 17372 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:22.198757 17380 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:03:22.205286 17380 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:03:22.215355 17380 catalog_manager.cc:1347] Generated new cluster ID: f9483d3e6dd14dada993025cc2caf674
I20260926 15:03:22.215492 17380 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:03:22.241926 17380 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:03:22.243886 17380 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:03:22.255187 17380 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07: Generated new TSK 0
I20260926 15:03:22.256106 17380 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.123.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.123.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.123.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.65.keytab.
Entry for principal oryx/127.9.123.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.65.keytab.
I20260926 15:03:22.316880 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:41723
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.65.keytab
--principal=oryx/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:36137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:03:22.655120 17391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:22.655440 17391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:22.655568 17391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:22.663060 17391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:22.663203 17391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:22.663305 17391 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 15:03:22.663419 17391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:03:22.673674 17391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:36137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.65.keytab
--principal=oryx/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.123.126:41723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:22.674998 17391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:22.676327 17391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:22.690228 17399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:22.690307 17396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:22.690300 17397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:22.691020 17391 server_base.cc:1034] running on GCE node
I20260926 15:03:22.691645 17391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:22.692593 17391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:22.693864 17391 hybrid_clock.cc:648] HybridClock initialized: now 1790435002693790 us; error 44 us; skew 500 ppm
I20260926 15:03:22.694196 17391 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:22 dist-test-slave-7153 krb5kdc[17275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435002, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:22.697480 17391 init.cc:377] Logged in from keytab as oryx/127.9.123.65@KRBTEST.COM (short username oryx)
I20260926 15:03:22.699028 17391 webserver.cc:460] Webserver started at http://127.9.123.65:43007/ using document root <none> and password file <none>
I20260926 15:03:22.699764 17391 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:22.699867 17391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:22.700168 17391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:22.703202 17391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "e5a1a54cb63c445380be679d749f8c62"
format_stamp: "Formatted at 2026-09-26 15:03:22 on dist-test-slave-7153"
server_key: "9777644c34f346af7f4e5ae220e9ba25"
server_key_iv: "33c570889331f8982f14b961f29019e8"
server_key_version: "encryptionkey@0"
I20260926 15:03:22.704037 17391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "e5a1a54cb63c445380be679d749f8c62"
format_stamp: "Formatted at 2026-09-26 15:03:22 on dist-test-slave-7153"
server_key: "9777644c34f346af7f4e5ae220e9ba25"
server_key_iv: "33c570889331f8982f14b961f29019e8"
server_key_version: "encryptionkey@0"
I20260926 15:03:22.709746 17391 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 15:03:22.713856 17406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:22.715315 17391 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:03:22.715520 17391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "e5a1a54cb63c445380be679d749f8c62"
format_stamp: "Formatted at 2026-09-26 15:03:22 on dist-test-slave-7153"
server_key: "9777644c34f346af7f4e5ae220e9ba25"
server_key_iv: "33c570889331f8982f14b961f29019e8"
server_key_version: "encryptionkey@0"
I20260926 15:03:22.715749 17391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:22.733013 17391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:22.737584 17391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:22.737859 17391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:22.740702 17391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:22.740813 17391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:22.740963 17391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:22.741123 17391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Sep 26 15:03:22 dist-test-slave-7153 krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435002, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM
I20260926 15:03:22.764286 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:22.741197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:36661 (local address 127.9.123.126:41723)
0926 15:03:22.741695 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:22.741702 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:22.742976 (+ 1274us) server_negotiation.cc:407] Connection header received
0926 15:03:22.745173 (+ 2197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:22.745178 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:22.745273 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:22.745367 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:22.747822 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:22.748776 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:22.751848 (+ 3072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:22.752455 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:22.752605 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:22.757796 (+ 5191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:22.757805 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:22.757811 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:22.757860 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:22.760819 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:22.761932 (+ 1113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:22.761937 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:22.761939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:22.762064 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:22.762509 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:22.762513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:22.762515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:22.762793 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:22.762920 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:22.763700 (+ 780us) server_negotiation.cc:299] Negotiation successful
0926 15:03:22.763929 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":273,"threads_started":1}
I20260926 15:03:22.765640 17416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:22.741973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:41723 (local address 127.9.123.65:36661)
0926 15:03:22.742808 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:22.742852 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:22.744821 (+ 1969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:22.745685 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:22.745698 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:22.746553 (+ 855us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:22.747605 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:22.747637 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:22.749002 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:22.749011 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:22.751602 (+ 2591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:22.751626 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:22.752976 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:22.752987 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:22.753304 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:22.754038 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:22.754064 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:22.757433 (+ 3369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:22.760996 (+ 3563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:22.761010 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:22.761032 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:22.761759 (+ 727us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:22.762175 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:22.762180 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:22.762184 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:22.762389 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:22.762918 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:22.762927 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:22.763331 (+ 404us) client_negotiation.cc:769] Sending connection context
0926 15:03:22.763583 (+ 252us) client_negotiation.cc:240] Negotiation successful
0926 15:03:22.764141 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":333,"threads_started":1}
I20260926 15:03:22.786171 17526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:33917 every 8 connection(s)
I20260926 15:03:22.786206 17391 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:33917
I20260926 15:03:22.787796 17391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:03:22.788992 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:22.773376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:57485 (local address 127.9.123.126:41723)
0926 15:03:22.773563 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:22.773569 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:22.774191 (+ 622us) server_negotiation.cc:407] Connection header received
0926 15:03:22.774368 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:22.774374 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:22.774471 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:22.774574 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:22.776107 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:22.777216 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:22.779138 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:22.779810 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:22.779921 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:22.782899 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:22.782907 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:22.782911 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:22.782959 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:22.786221 (+ 3262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:22.786897 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:22.786903 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:22.786905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:22.787053 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:22.787479 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:22.787484 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:22.787487 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:22.787810 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:22.787966 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:22.788376 (+ 410us) server_negotiation.cc:299] Negotiation successful
0926 15:03:22.788623 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 15:03:22.789134 17489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:22.773053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:41723 (local address 127.9.123.65:57485)
0926 15:03:22.774079 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:22.774096 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:22.774248 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:22.774643 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:22.774649 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:22.775349 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:22.775918 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:22.775940 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:22.777380 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:22.777386 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:22.778911 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:22.778933 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:22.780296 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:22.780305 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:22.780539 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:22.781358 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:22.781387 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:22.782643 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:22.786366 (+ 3723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:22.786372 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:22.786376 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:22.786783 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:22.787202 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:22.787206 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:22.787209 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:22.787362 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:22.787954 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:22.787959 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:22.788119 (+ 160us) client_negotiation.cc:769] Sending connection context
0926 15:03:22.788248 (+ 129us) client_negotiation.cc:240] Negotiation successful
0926 15:03:22.788638 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"thread_start_us":683,"threads_started":1}
I20260926 15:03:22.789958 17527 heartbeater.cc:347] Connected to a master server at 127.9.123.126:41723
I20260926 15:03:22.790182 17527 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:22.790663 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 17391
I20260926 15:03:22.790849 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:03:22.790845 17527 heartbeater.cc:511] Master 127.9.123.126:41723 requested a full tablet report, sending...
I20260926 15:03:22.791658 9709 external_mini_cluster.cc:1428] Setting key bd5d4e661ed96c85556470c80ac3900f
I20260926 15:03:22.793155 17320 ts_manager.cc:194] Registered new tserver with Master: e5a1a54cb63c445380be679d749f8c62 (127.9.123.65:33917)
I20260926 15:03:22.794701 17320 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.123.65@KRBTEST.COM'} at 127.9.123.65:36661
WARNING: no policy specified for oryx/127.9.123.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.123.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.123.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.66.keytab.
Entry for principal oryx/127.9.123.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.66.keytab.
I20260926 15:03:22.840711 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:41723
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.66.keytab
--principal=oryx/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:36137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:03:23.171823 17531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:23.172205 17531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:23.172363 17531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:23.180893 17531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:23.181017 17531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:23.181124 17531 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 15:03:23.181259 17531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:03:23.191150 17531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:36137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.66.keytab
--principal=oryx/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.123.126:41723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:23.192438 17531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:23.193853 17531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:23.208269 17537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:23.208343 17536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:23.208284 17539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:23.209061 17531 server_base.cc:1034] running on GCE node
I20260926 15:03:23.209842 17531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:23.211027 17531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:23.212275 17531 hybrid_clock.cc:648] HybridClock initialized: now 1790435003212191 us; error 62 us; skew 500 ppm
I20260926 15:03:23.212618 17531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:23 dist-test-slave-7153 krb5kdc[17275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435003, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:23.215852 17531 init.cc:377] Logged in from keytab as oryx/127.9.123.66@KRBTEST.COM (short username oryx)
I20260926 15:03:23.217639 17531 webserver.cc:460] Webserver started at http://127.9.123.66:40395/ using document root <none> and password file <none>
I20260926 15:03:23.218544 17531 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:23.218658 17531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:23.219024 17531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:23.222347 17531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "4ae03fe546924cad854258eb297dc52a"
format_stamp: "Formatted at 2026-09-26 15:03:23 on dist-test-slave-7153"
server_key: "a37e700ef9ee4ff9a01a43ae61aa19e2"
server_key_iv: "e1b98203cbc28e0a15f1175e7c252254"
server_key_version: "encryptionkey@0"
I20260926 15:03:23.223197 17531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "4ae03fe546924cad854258eb297dc52a"
format_stamp: "Formatted at 2026-09-26 15:03:23 on dist-test-slave-7153"
server_key: "a37e700ef9ee4ff9a01a43ae61aa19e2"
server_key_iv: "e1b98203cbc28e0a15f1175e7c252254"
server_key_version: "encryptionkey@0"
I20260926 15:03:23.228533 17531 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 15:03:23.232386 17546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:23.234019 17531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260926 15:03:23.234194 17531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "4ae03fe546924cad854258eb297dc52a"
format_stamp: "Formatted at 2026-09-26 15:03:23 on dist-test-slave-7153"
server_key: "a37e700ef9ee4ff9a01a43ae61aa19e2"
server_key_iv: "e1b98203cbc28e0a15f1175e7c252254"
server_key_version: "encryptionkey@0"
I20260926 15:03:23.234423 17531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:23.255438 17531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:23.259461 17531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:23.259714 17531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:23.262428 17531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:23.262524 17531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:23.262660 17531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:23.262765 17531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Sep 26 15:03:23 dist-test-slave-7153 krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435003, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM
I20260926 15:03:23.286094 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:23.262930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:50617 (local address 127.9.123.126:41723)
0926 15:03:23.263146 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:23.263154 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:23.264617 (+ 1463us) server_negotiation.cc:407] Connection header received
0926 15:03:23.266795 (+ 2178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:23.266800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:23.266891 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:23.266989 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:23.269933 (+ 2944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:23.270906 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:23.273266 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:23.273943 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:23.274088 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:23.279049 (+ 4961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:23.279055 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:23.279059 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:23.279096 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:23.282291 (+ 3195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:23.283382 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:23.283387 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:23.283389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:23.283532 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:23.284095 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:23.284101 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:23.284104 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:23.284419 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:23.284545 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:23.285636 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0926 15:03:23.285846 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 15:03:23.287838 17555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:23.263595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:41723 (local address 127.9.123.66:50617)
0926 15:03:23.264422 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:23.264470 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:23.266490 (+ 2020us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:23.267519 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:23.267537 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:23.268548 (+ 1011us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:23.269673 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:23.269710 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:23.271122 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:23.271128 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:23.273068 (+ 1940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:23.273086 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:23.274216 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:23.274223 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:23.274425 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:23.275329 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:23.275365 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:23.278707 (+ 3342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:23.282486 (+ 3779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:23.282500 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:23.282524 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:23.283204 (+ 680us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:23.283636 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:23.283641 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:23.283645 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:23.283938 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:23.284547 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:23.284558 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:23.285122 (+ 564us) client_negotiation.cc:769] Sending connection context
0926 15:03:23.285513 (+ 391us) client_negotiation.cc:240] Negotiation successful
0926 15:03:23.286147 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":341,"threads_started":1}
I20260926 15:03:23.311043 17531 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:34747
I20260926 15:03:23.311066 17665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:34747 every 8 connection(s)
I20260926 15:03:23.312362 17531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:03:23.314677 17666 heartbeater.cc:347] Connected to a master server at 127.9.123.126:41723
I20260926 15:03:23.314950 17666 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:23.315379 17649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:23.300708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:41723 (local address 127.9.123.66:51805)
0926 15:03:23.301427 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:23.301449 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:23.301618 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:23.301931 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:23.301936 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:23.302545 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:23.302925 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:23.302938 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:23.303910 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:23.303915 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:23.305654 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:23.305676 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:23.306611 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:23.306619 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:23.306801 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:23.307544 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:23.307569 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:23.308725 (+ 1156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:23.312179 (+ 3454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:23.312184 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:23.312188 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:23.312746 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:23.313213 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:23.313220 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:23.313224 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:23.313406 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:23.314120 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:23.314124 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:23.314317 (+ 193us) client_negotiation.cc:769] Sending connection context
0926 15:03:23.314435 (+ 118us) client_negotiation.cc:240] Negotiation successful
0926 15:03:23.314797 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":341,"threads_started":1}
I20260926 15:03:23.315706 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:23.300729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:51805 (local address 127.9.123.126:41723)
0926 15:03:23.300916 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:23.300922 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:23.301547 (+ 625us) server_negotiation.cc:407] Connection header received
0926 15:03:23.301743 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:23.301747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:23.301810 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:23.301881 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:23.303076 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:23.303782 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:23.305869 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:23.306401 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:23.306543 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:23.308992 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:23.308997 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:23.309000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:23.309033 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:23.312043 (+ 3010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:23.312890 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:23.312897 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:23.312900 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:23.313073 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:23.313533 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:23.313538 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:23.313541 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:23.313971 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:23.315078 (+ 1107us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:23.315202 (+ 124us) server_negotiation.cc:299] Negotiation successful
0926 15:03:23.315428 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 15:03:23.316046 17666 heartbeater.cc:511] Master 127.9.123.126:41723 requested a full tablet report, sending...
I20260926 15:03:23.317366 17320 ts_manager.cc:194] Registered new tserver with Master: 4ae03fe546924cad854258eb297dc52a (127.9.123.66:34747)
I20260926 15:03:23.318295 17320 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.123.66@KRBTEST.COM'} at 127.9.123.66:50617
I20260926 15:03:23.321475 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 17531
I20260926 15:03:23.321620 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:03:23.322039 9709 external_mini_cluster.cc:1428] Setting key 89545a24d3c465d38a3069844b8033c8
WARNING: no policy specified for oryx/127.9.123.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.123.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.123.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.67.keytab.
Entry for principal oryx/127.9.123.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.67.keytab.
I20260926 15:03:23.368657 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:41723
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.67.keytab
--principal=oryx/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:36137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:03:23.694352 17670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:23.694639 17670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:23.694762 17670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:23.703090 17670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:23.703234 17670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:23.703339 17670 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260926 15:03:23.703442 17670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:03:23.714145 17670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:36137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.67.keytab
--principal=oryx/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.123.126:41723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:23.715497 17670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:23.716919 17670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:23.731455 17676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:23.731775 17675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:23.731809 17678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:23.732414 17670 server_base.cc:1034] running on GCE node
I20260926 15:03:23.733067 17670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:23.734170 17670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:23.735532 17670 hybrid_clock.cc:648] HybridClock initialized: now 1790435003735443 us; error 60 us; skew 500 ppm
I20260926 15:03:23.735895 17670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:23 dist-test-slave-7153 krb5kdc[17275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435003, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:23.739464 17670 init.cc:377] Logged in from keytab as oryx/127.9.123.67@KRBTEST.COM (short username oryx)
I20260926 15:03:23.741138 17670 webserver.cc:460] Webserver started at http://127.9.123.67:42097/ using document root <none> and password file <none>
I20260926 15:03:23.741886 17670 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:23.741999 17670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:23.742328 17670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:23.745625 17670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "e9d05cc314cb404086e36463718888a5"
format_stamp: "Formatted at 2026-09-26 15:03:23 on dist-test-slave-7153"
server_key: "437c7856c846daab34022c346d8f96a7"
server_key_iv: "6b6438044916bbe4ff9877b376eef335"
server_key_version: "encryptionkey@0"
I20260926 15:03:23.746518 17670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "e9d05cc314cb404086e36463718888a5"
format_stamp: "Formatted at 2026-09-26 15:03:23 on dist-test-slave-7153"
server_key: "437c7856c846daab34022c346d8f96a7"
server_key_iv: "6b6438044916bbe4ff9877b376eef335"
server_key_version: "encryptionkey@0"
I20260926 15:03:23.752451 17670 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.002s
I20260926 15:03:23.757447 17685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:23.759029 17670 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 15:03:23.759227 17670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "e9d05cc314cb404086e36463718888a5"
format_stamp: "Formatted at 2026-09-26 15:03:23 on dist-test-slave-7153"
server_key: "437c7856c846daab34022c346d8f96a7"
server_key_iv: "6b6438044916bbe4ff9877b376eef335"
server_key_version: "encryptionkey@0"
I20260926 15:03:23.759478 17670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:23.788081 17670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:23.793496 17670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:23.793737 17670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:23.796381 17670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:23.796487 17670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:23.796626 17670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:23.796681 17670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:23.797574 17527 heartbeater.cc:503] Master 127.9.123.126:41723 was elected leader, sending a full tablet report...
Sep 26 15:03:23 dist-test-slave-7153 krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435003, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM
I20260926 15:03:23.825935 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:23.797277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:48691 (local address 127.9.123.126:41723)
0926 15:03:23.797460 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:23.797467 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:23.799224 (+ 1757us) server_negotiation.cc:407] Connection header received
0926 15:03:23.802244 (+ 3020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:23.802251 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:23.802347 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:23.802459 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:23.804672 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:23.805607 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:23.808024 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:23.808518 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:23.808655 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:23.817488 (+ 8833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:23.817496 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:23.817501 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:23.817576 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:23.821340 (+ 3764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:23.822697 (+ 1357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:23.822703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:23.822707 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:23.822888 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:23.823523 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:23.823528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:23.823532 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:23.823913 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:23.824068 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:23.825284 (+ 1216us) server_negotiation.cc:299] Negotiation successful
0926 15:03:23.825569 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 15:03:23.827724 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:23.797999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:41723 (local address 127.9.123.67:48691)
0926 15:03:23.798980 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:23.799063 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:23.801846 (+ 2783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:23.802746 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:23.802756 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:23.803529 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:23.804388 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:23.804414 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:23.805861 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:23.805865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:23.807853 (+ 1988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:23.807871 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:23.808806 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:23.808814 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:23.809199 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:23.810131 (+ 932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:23.810164 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:23.817120 (+ 6956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:23.821578 (+ 4458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:23.821595 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:23.821619 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:23.822479 (+ 860us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:23.823043 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:23.823050 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:23.823057 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:23.823380 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:23.824074 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:23.824085 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:23.824722 (+ 637us) client_negotiation.cc:769] Sending connection context
0926 15:03:23.825144 (+ 422us) client_negotiation.cc:240] Negotiation successful
0926 15:03:23.825800 (+ 656us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":366,"threads_started":1}
I20260926 15:03:23.838244 17670 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:42353
I20260926 15:03:23.838301 17804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:42353 every 8 connection(s)
I20260926 15:03:23.839633 17670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:03:23.840533 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 17670
I20260926 15:03:23.840700 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:03:23.841355 9709 external_mini_cluster.cc:1428] Setting key 6956527ce26cf0811e28061e47a5bc8d
I20260926 15:03:23.841925 17805 heartbeater.cc:347] Connected to a master server at 127.9.123.126:41723
I20260926 15:03:23.842161 17805 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:23.843091 17805 heartbeater.cc:511] Master 127.9.123.126:41723 requested a full tablet report, sending...
I20260926 15:03:23.844781 17320 ts_manager.cc:194] Registered new tserver with Master: e9d05cc314cb404086e36463718888a5 (127.9.123.67:42353)
I20260926 15:03:23.845813 17320 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.123.67@KRBTEST.COM'} at 127.9.123.67:48691
I20260926 15:03:23.851590 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:23.835939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:40751 (local address 127.9.123.126:41723)
0926 15:03:23.836120 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:23.836127 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:23.836722 (+ 595us) server_negotiation.cc:407] Connection header received
0926 15:03:23.836900 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:23.836907 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:23.836984 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:23.837080 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:23.838674 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:23.839443 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:23.841546 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:23.842242 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:23.842374 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:23.846460 (+ 4086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:23.846465 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:23.846468 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:23.846498 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:23.849043 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:23.849749 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:23.849755 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:23.849758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:23.849890 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:23.850238 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:23.850242 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:23.850244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:23.850500 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:23.850638 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:23.851032 (+ 394us) server_negotiation.cc:299] Negotiation successful
0926 15:03:23.851243 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 15:03:23.851649 17800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:23.835997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:41723 (local address 127.9.123.67:40751)
0926 15:03:23.836606 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:23.836627 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:23.836770 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:23.837243 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:23.837248 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:23.837968 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:23.838523 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:23.838543 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:23.839573 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:23.839578 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:23.841345 (+ 1767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:23.841363 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:23.842701 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:23.842708 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:23.842941 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:23.845306 (+ 2365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:23.845329 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:23.846246 (+ 917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:23.849211 (+ 2965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:23.849217 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:23.849221 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:23.849615 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:23.850000 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:23.850005 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:23.850009 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:23.850129 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:23.850611 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:23.850616 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:23.850783 (+ 167us) client_negotiation.cc:769] Sending connection context
0926 15:03:23.850898 (+ 115us) client_negotiation.cc:240] Negotiation successful
0926 15:03:23.851201 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":346,"threads_started":1}
I20260926 15:03:23.857179 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 15:03:23 dist-test-slave-7153 krb5kdc[17275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435003, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 26 15:03:23 dist-test-slave-7153 krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:23 dist-test-slave-7153 krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:24.320855 17666 heartbeater.cc:503] Master 127.9.123.126:41723 was elected leader, sending a full tablet report...
I20260926 15:03:24.848843 17805 heartbeater.cc:503] Master 127.9.123.126:41723 was elected leader, sending a full tablet report...
W20260926 15:03:24.874269 17813 sasl_common.cc:323] Time spent calling sasl_client_start(): real 0.973s user 0.006s sys 0.007s
W20260926 15:03:24.875434 17414 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:23.894796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37576 (local address 127.9.123.126:41723)
0926 15:03:23.894975 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:23.894981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:23.895310 (+ 329us) server_negotiation.cc:407] Connection header received
0926 15:03:23.895443 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:23.895448 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:23.895544 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:23.895635 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:23.897231 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:23.898113 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:23.899498 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:23.899954 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:23.900057 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:24.875125 (+975068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37576: BlockingRecv error: recv got EOF from 127.0.0.1:37576 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 26 15:03:24 dist-test-slave-7153 krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435003, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM
I20260926 15:03:24.899646 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:24.879098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37584 (local address 127.9.123.126:41723)
0926 15:03:24.879243 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:24.879248 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:24.879580 (+ 332us) server_negotiation.cc:407] Connection header received
0926 15:03:24.879723 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:24.879727 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:24.879806 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:24.879895 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:24.881145 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:24.881877 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:24.883186 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:24.883604 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:24.883699 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:24.894749 (+ 11050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:24.894754 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:24.894758 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:24.894800 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:24.897410 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:24.898121 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:24.898125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:24.898127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:24.898238 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:24.898559 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:24.898562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:24.898564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:24.898811 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:24.899043 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:24.899248 (+ 205us) server_negotiation.cc:299] Negotiation successful
0926 15:03:24.899426 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260926 15:03:24.903733 17320 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37584:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:03:24.910295 17320 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:03:24.933540 17824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:24.922408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:42353 (local address 127.0.0.1:60510)
0926 15:03:24.923267 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:24.923315 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:24.923490 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:24.925810 (+ 2320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:24.925816 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:24.925853 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:24.926455 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:24.926467 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:24.928269 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:24.928274 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:24.930550 (+ 2276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:24.930564 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:24.932436 (+ 1872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:24.932442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:24.932617 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:24.932681 (+ 64us) client_negotiation.cc:769] Sending connection context
0926 15:03:24.932806 (+ 125us) client_negotiation.cc:240] Negotiation successful
0926 15:03:24.933010 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"mutex_wait_us":84,"thread_start_us":350,"threads_started":1}
I20260926 15:03:24.933702 17825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:24.922638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60510 (local address 127.9.123.67:42353)
0926 15:03:24.925210 (+ 2572us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:24.925218 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:24.925245 (+ 27us) server_negotiation.cc:407] Connection header received
0926 15:03:24.925336 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:24.925344 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:24.925576 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:24.925772 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:24.926622 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:24.928123 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:24.930694 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:24.932241 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:24.932372 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:24.932584 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:24.933261 (+ 677us) server_negotiation.cc:299] Negotiation successful
0926 15:03:24.933434 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2404,"thread_start_us":2287,"threads_started":1}
I20260926 15:03:24.933988 17829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:24.924146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37844 (local address 127.9.123.65:33917)
0926 15:03:24.924797 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:24.924805 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:24.926656 (+ 1851us) server_negotiation.cc:407] Connection header received
0926 15:03:24.926809 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:24.926816 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:24.927052 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:24.927247 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:24.927974 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:24.929423 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:24.931666 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:24.932698 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:24.932829 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:24.933020 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:24.933551 (+ 531us) server_negotiation.cc:299] Negotiation successful
0926 15:03:24.933720 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":349,"threads_started":1}
I20260926 15:03:24.934199 17827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:24.923503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55700 (local address 127.9.123.66:34747)
0926 15:03:24.924072 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:24.924082 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:24.924590 (+ 508us) server_negotiation.cc:407] Connection header received
0926 15:03:24.924910 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:24.924919 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:24.925227 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:24.925403 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:24.927320 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:24.928288 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:24.930769 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:24.932235 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:24.932372 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:24.932584 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:24.933797 (+ 1213us) server_negotiation.cc:299] Negotiation successful
0926 15:03:24.933958 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":309,"threads_started":1}
I20260926 15:03:24.934872 17826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:24.922953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:34747 (local address 127.0.0.1:55700)
0926 15:03:24.924478 (+ 1525us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:24.924496 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:24.924690 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:24.925677 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:24.925685 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:24.925715 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:24.926337 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:24.926355 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:24.928431 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:24.928436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:24.930623 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:24.930641 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:24.932721 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:24.932726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:24.932876 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:24.932913 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 15:03:24.932993 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 15:03:24.933143 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1417,"mutex_wait_us":1,"thread_start_us":1296,"threads_started":1}
I20260926 15:03:24.934934 17828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:24.923703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:33917 (local address 127.0.0.1:37844)
0926 15:03:24.926557 (+ 2854us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:24.926573 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:24.926676 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:24.927443 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:24.927448 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:24.927470 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:24.927785 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:24.927795 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:24.929595 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:24.929600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:24.931524 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:24.931534 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:24.933149 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:24.933156 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:24.933332 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:24.933366 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 15:03:24.934504 (+ 1138us) client_negotiation.cc:240] Negotiation successful
0926 15:03:24.934643 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2736,"thread_start_us":2420,"threads_started":1}
I20260926 15:03:24.937840 17596 tablet_service.cc:1467] Processing CreateTablet for tablet 3b52c4dbcca14cb6aca4914e38f45035 (DEFAULT_TABLE table=test-table [id=a2599273b1d641e6b774666e52c78cc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:03:24.937891 17457 tablet_service.cc:1467] Processing CreateTablet for tablet 3b52c4dbcca14cb6aca4914e38f45035 (DEFAULT_TABLE table=test-table [id=a2599273b1d641e6b774666e52c78cc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:03:24.937908 17735 tablet_service.cc:1467] Processing CreateTablet for tablet 3b52c4dbcca14cb6aca4914e38f45035 (DEFAULT_TABLE table=test-table [id=a2599273b1d641e6b774666e52c78cc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:03:24.939220 17596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b52c4dbcca14cb6aca4914e38f45035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:24.939218 17735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b52c4dbcca14cb6aca4914e38f45035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:24.939219 17457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b52c4dbcca14cb6aca4914e38f45035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:24.952055 17830 tablet_bootstrap.cc:492] T 3b52c4dbcca14cb6aca4914e38f45035 P 4ae03fe546924cad854258eb297dc52a: Bootstrap starting.
I20260926 15:03:24.953609 17831 tablet_bootstrap.cc:492] T 3b52c4dbcca14cb6aca4914e38f45035 P e9d05cc314cb404086e36463718888a5: Bootstrap starting.
I20260926 15:03:24.954340 17832 tablet_bootstrap.cc:492] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62: Bootstrap starting.
I20260926 15:03:24.955530 17830 tablet_bootstrap.cc:654] T 3b52c4dbcca14cb6aca4914e38f45035 P 4ae03fe546924cad854258eb297dc52a: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:24.956880 17830 log.cc:824] T 3b52c4dbcca14cb6aca4914e38f45035 P 4ae03fe546924cad854258eb297dc52a: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:24.957885 17832 tablet_bootstrap.cc:654] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:24.959337 17832 log.cc:824] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:24.960101 17831 tablet_bootstrap.cc:654] T 3b52c4dbcca14cb6aca4914e38f45035 P e9d05cc314cb404086e36463718888a5: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:24.961426 17831 log.cc:824] T 3b52c4dbcca14cb6aca4914e38f45035 P e9d05cc314cb404086e36463718888a5: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:24.962440 17832 tablet_bootstrap.cc:492] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62: No bootstrap required, opened a new log
I20260926 15:03:24.962452 17830 tablet_bootstrap.cc:492] T 3b52c4dbcca14cb6aca4914e38f45035 P 4ae03fe546924cad854258eb297dc52a: No bootstrap required, opened a new log
I20260926 15:03:24.962725 17832 ts_tablet_manager.cc:1397] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 15:03:24.962725 17830 ts_tablet_manager.cc:1397] T 3b52c4dbcca14cb6aca4914e38f45035 P 4ae03fe546924cad854258eb297dc52a: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260926 15:03:24.964424 17831 tablet_bootstrap.cc:492] T 3b52c4dbcca14cb6aca4914e38f45035 P e9d05cc314cb404086e36463718888a5: No bootstrap required, opened a new log
I20260926 15:03:24.964751 17831 ts_tablet_manager.cc:1397] T 3b52c4dbcca14cb6aca4914e38f45035 P e9d05cc314cb404086e36463718888a5: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260926 15:03:24.970451 17832 raft_consensus.cc:348] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } } peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } }
I20260926 15:03:24.970870 17832 raft_consensus.cc:374] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:24.970988 17832 raft_consensus.cc:729] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5a1a54cb63c445380be679d749f8c62, State: Initialized, Role: FOLLOWER
I20260926 15:03:24.971614 17832 consensus_queue.cc:260] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } } peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } }
I20260926 15:03:24.973013 17830 raft_consensus.cc:348] T 3b52c4dbcca14cb6aca4914e38f45035 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } } peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } }
I20260926 15:03:24.973428 17832 ts_tablet_manager.cc:1428] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260926 15:03:24.973554 17830 raft_consensus.cc:374] T 3b52c4dbcca14cb6aca4914e38f45035 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:24.973701 17830 raft_consensus.cc:729] T 3b52c4dbcca14cb6aca4914e38f45035 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ae03fe546924cad854258eb297dc52a, State: Initialized, Role: FOLLOWER
I20260926 15:03:24.974012 17837 raft_consensus.cc:482] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:03:24.974458 17830 consensus_queue.cc:260] T 3b52c4dbcca14cb6aca4914e38f45035 P 4ae03fe546924cad854258eb297dc52a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } } peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } }
I20260926 15:03:24.974321 17837 raft_consensus.cc:504] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } } peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } }
I20260926 15:03:24.975107 17831 raft_consensus.cc:348] T 3b52c4dbcca14cb6aca4914e38f45035 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } } peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } }
I20260926 15:03:24.975638 17831 raft_consensus.cc:374] T 3b52c4dbcca14cb6aca4914e38f45035 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:24.975768 17831 raft_consensus.cc:729] T 3b52c4dbcca14cb6aca4914e38f45035 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d05cc314cb404086e36463718888a5, State: Initialized, Role: FOLLOWER
I20260926 15:03:24.976397 17831 consensus_queue.cc:260] T 3b52c4dbcca14cb6aca4914e38f45035 P e9d05cc314cb404086e36463718888a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } } peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } }
I20260926 15:03:24.977416 17830 ts_tablet_manager.cc:1428] T 3b52c4dbcca14cb6aca4914e38f45035 P 4ae03fe546924cad854258eb297dc52a: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260926 15:03:24.978201 17837 leader_election.cc:288] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9d05cc314cb404086e36463718888a5 (127.9.123.67:42353), 4ae03fe546924cad854258eb297dc52a (127.9.123.66:34747)
I20260926 15:03:24.984078 17831 ts_tablet_manager.cc:1428] T 3b52c4dbcca14cb6aca4914e38f45035 P e9d05cc314cb404086e36463718888a5: Time spent starting tablet: real 0.019s user 0.007s sys 0.008s
I20260926 15:03:24.992884 17827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:24.978937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:34425 (local address 127.9.123.66:34747)
0926 15:03:24.979115 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:24.979121 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:24.987308 (+ 8187us) server_negotiation.cc:407] Connection header received
0926 15:03:24.987409 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:24.987413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:24.987482 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:24.987554 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:24.988152 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:24.988969 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:24.991023 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:24.992047 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:24.992160 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:24.992213 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:24.992599 (+ 386us) server_negotiation.cc:299] Negotiation successful
0926 15:03:24.992686 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 15:03:24.993058 17840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:24.979032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:34747 (local address 127.9.123.65:34425)
0926 15:03:24.987168 (+ 8136us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:24.987184 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:24.987315 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:24.987606 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:24.987611 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:24.987644 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:24.987988 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:24.988001 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:24.989087 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:24.989093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:24.990885 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:24.990898 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:24.992262 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:24.992270 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:24.992442 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:24.992478 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 15:03:24.992576 (+ 98us) client_negotiation.cc:240] Negotiation successful
0926 15:03:24.992709 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8010,"thread_start_us":7523,"threads_started":1}
I20260926 15:03:24.993847 17825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:24.979493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:41777 (local address 127.9.123.67:42353)
0926 15:03:24.979679 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:24.979684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:24.986964 (+ 7280us) server_negotiation.cc:407] Connection header received
0926 15:03:24.988197 (+ 1233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:24.988202 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:24.988272 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:24.988344 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:24.988824 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:24.989817 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:24.991810 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:24.992853 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:24.992986 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:24.993052 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:24.993508 (+ 456us) server_negotiation.cc:299] Negotiation successful
0926 15:03:24.993631 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260926 15:03:24.993925 17841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:24.978460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:42353 (local address 127.9.123.65:41777)
0926 15:03:24.986852 (+ 8392us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:24.986877 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:24.987267 (+ 390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:24.988381 (+ 1114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:24.988385 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:24.988418 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:24.988685 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:24.988693 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:24.989935 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:24.989939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:24.991630 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:24.991644 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:24.993079 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:24.993084 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:24.993301 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:24.993334 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 15:03:24.993469 (+ 135us) client_negotiation.cc:240] Negotiation successful
0926 15:03:24.993597 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8235,"thread_start_us":8480,"threads_started":1}
I20260926 15:03:24.994071 17616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b52c4dbcca14cb6aca4914e38f45035" candidate_uuid: "e5a1a54cb63c445380be679d749f8c62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ae03fe546924cad854258eb297dc52a" is_pre_election: true
I20260926 15:03:24.994647 17616 raft_consensus.cc:2454] T 3b52c4dbcca14cb6aca4914e38f45035 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5a1a54cb63c445380be679d749f8c62 in term 0.
I20260926 15:03:24.994887 17755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b52c4dbcca14cb6aca4914e38f45035" candidate_uuid: "e5a1a54cb63c445380be679d749f8c62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d05cc314cb404086e36463718888a5" is_pre_election: true
I20260926 15:03:24.995299 17408 leader_election.cc:302] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ae03fe546924cad854258eb297dc52a, e5a1a54cb63c445380be679d749f8c62; no voters:
I20260926 15:03:24.995332 17755 raft_consensus.cc:2454] T 3b52c4dbcca14cb6aca4914e38f45035 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5a1a54cb63c445380be679d749f8c62 in term 0.
I20260926 15:03:24.995658 17837 raft_consensus.cc:2781] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:03:24.995792 17837 raft_consensus.cc:482] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:03:24.995911 17837 raft_consensus.cc:3037] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:24.998415 17837 raft_consensus.cc:504] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } } peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } }
I20260926 15:03:24.999051 17837 leader_election.cc:288] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62 [CANDIDATE]: Term 1 election: Requested vote from peers e9d05cc314cb404086e36463718888a5 (127.9.123.67:42353), 4ae03fe546924cad854258eb297dc52a (127.9.123.66:34747)
I20260926 15:03:24.999485 17755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b52c4dbcca14cb6aca4914e38f45035" candidate_uuid: "e5a1a54cb63c445380be679d749f8c62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d05cc314cb404086e36463718888a5"
I20260926 15:03:24.999516 17616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b52c4dbcca14cb6aca4914e38f45035" candidate_uuid: "e5a1a54cb63c445380be679d749f8c62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ae03fe546924cad854258eb297dc52a"
I20260926 15:03:24.999706 17755 raft_consensus.cc:3037] T 3b52c4dbcca14cb6aca4914e38f45035 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:24.999706 17616 raft_consensus.cc:3037] T 3b52c4dbcca14cb6aca4914e38f45035 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:25.002875 17616 raft_consensus.cc:2454] T 3b52c4dbcca14cb6aca4914e38f45035 P 4ae03fe546924cad854258eb297dc52a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5a1a54cb63c445380be679d749f8c62 in term 1.
I20260926 15:03:25.002875 17755 raft_consensus.cc:2454] T 3b52c4dbcca14cb6aca4914e38f45035 P e9d05cc314cb404086e36463718888a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5a1a54cb63c445380be679d749f8c62 in term 1.
I20260926 15:03:25.003326 17408 leader_election.cc:302] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ae03fe546924cad854258eb297dc52a, e5a1a54cb63c445380be679d749f8c62; no voters:
I20260926 15:03:25.003630 17837 raft_consensus.cc:2781] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:03:25.003805 17837 raft_consensus.cc:686] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62 [term 1 LEADER]: Becoming Leader. State: Replica: e5a1a54cb63c445380be679d749f8c62, State: Running, Role: LEADER
I20260926 15:03:25.004261 17837 consensus_queue.cc:237] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } } peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } }
I20260926 15:03:25.008766 17320 catalog_manager.cc:5495] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62 reported cstate change: term changed from 0 to 1, leader changed from <none> to e5a1a54cb63c445380be679d749f8c62 (127.9.123.65). New cstate: current_term: 1 leader_uuid: "e5a1a54cb63c445380be679d749f8c62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } health_report { overall_health: UNKNOWN } } }
I20260926 15:03:25.026196 17829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:25.020728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37852 (local address 127.9.123.65:33917)
0926 15:03:25.021045 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:25.021051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:25.021068 (+ 17us) server_negotiation.cc:407] Connection header received
0926 15:03:25.021279 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:25.021283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:25.021361 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:25.021507 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:03:25.021857 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.022577 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.023947 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.024358 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.024470 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:25.024889 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:03:25.025521 (+ 632us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:03:25.025660 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:25.025796 (+ 136us) server_negotiation.cc:299] Negotiation successful
0926 15:03:25.025894 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176}
I20260926 15:03:25.035038 17755 raft_consensus.cc:1261] T 3b52c4dbcca14cb6aca4914e38f45035 P e9d05cc314cb404086e36463718888a5 [term 1 FOLLOWER]: Refusing update from remote peer e5a1a54cb63c445380be679d749f8c62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:03:25.035037 17616 raft_consensus.cc:1261] T 3b52c4dbcca14cb6aca4914e38f45035 P 4ae03fe546924cad854258eb297dc52a [term 1 FOLLOWER]: Refusing update from remote peer e5a1a54cb63c445380be679d749f8c62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:03:25.036006 17836 consensus_queue.cc:1035] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:03:25.036440 17837 consensus_queue.cc:1035] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260926 15:03:25.041383 17528 tablet.cc:2369] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:03:25.051651 17845 mvcc.cc:205] Tried to move back new op lower bound from 7333621780613115904 to 7333621780501626880. Current Snapshot: MvccSnapshot[applied={T|T < 7333621780613115904}]
I20260926 15:03:25.060887 17320 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:03:25.061160 17320 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:03:25.064966 17320 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07: Sending DeleteTablet for 3 replicas of tablet 3b52c4dbcca14cb6aca4914e38f45035
I20260926 15:03:25.066164 17596 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b52c4dbcca14cb6aca4914e38f45035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:03:25 UTC) from {username='oryx', principal='oryx/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:55700
I20260926 15:03:25.066181 17457 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b52c4dbcca14cb6aca4914e38f45035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:03:25 UTC) from {username='oryx', principal='oryx/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:37844
I20260926 15:03:25.067055 17854 tablet_replica.cc:330] T 3b52c4dbcca14cb6aca4914e38f45035 P 4ae03fe546924cad854258eb297dc52a: stopping tablet replica
I20260926 15:03:25.067107 17735 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b52c4dbcca14cb6aca4914e38f45035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:03:25 UTC) from {username='oryx', principal='oryx/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:60510
I20260926 15:03:25.067489 17854 raft_consensus.cc:2229] T 3b52c4dbcca14cb6aca4914e38f45035 P 4ae03fe546924cad854258eb297dc52a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:03:25.067974 17854 raft_consensus.cc:2258] T 3b52c4dbcca14cb6aca4914e38f45035 P 4ae03fe546924cad854258eb297dc52a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:25.068974 17856 tablet_replica.cc:330] T 3b52c4dbcca14cb6aca4914e38f45035 P e9d05cc314cb404086e36463718888a5: stopping tablet replica
I20260926 15:03:25.068797 17319 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37584:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:03:25.069583 17319 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:03:25.069602 17856 raft_consensus.cc:2229] T 3b52c4dbcca14cb6aca4914e38f45035 P e9d05cc314cb404086e36463718888a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:03:25.070080 17856 raft_consensus.cc:2258] T 3b52c4dbcca14cb6aca4914e38f45035 P e9d05cc314cb404086e36463718888a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:25.070480 17854 ts_tablet_manager.cc:1905] T 3b52c4dbcca14cb6aca4914e38f45035 P 4ae03fe546924cad854258eb297dc52a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:03:25.071076 17855 tablet_replica.cc:330] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62: stopping tablet replica
I20260926 15:03:25.071560 17855 raft_consensus.cc:2229] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62 [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:03:25.072276 17855 raft_consensus.cc:2258] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:25.076874 17457 tablet_service.cc:1467] Processing CreateTablet for tablet 0ba0a79ad7fb4e6a895cc46847525cb6 (DEFAULT_TABLE table=test-table [id=d98b0706549a4686af7766f0ee835384]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:03:25.077412 17457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ba0a79ad7fb4e6a895cc46847525cb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:25.077338 17735 tablet_service.cc:1467] Processing CreateTablet for tablet 0ba0a79ad7fb4e6a895cc46847525cb6 (DEFAULT_TABLE table=test-table [id=d98b0706549a4686af7766f0ee835384]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:03:25.077744 17735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ba0a79ad7fb4e6a895cc46847525cb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:25.078616 17854 ts_tablet_manager.cc:1918] T 3b52c4dbcca14cb6aca4914e38f45035 P 4ae03fe546924cad854258eb297dc52a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:03:25.078783 17854 log.cc:1196] T 3b52c4dbcca14cb6aca4914e38f45035 P 4ae03fe546924cad854258eb297dc52a: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/3b52c4dbcca14cb6aca4914e38f45035
I20260926 15:03:25.079326 17854 ts_tablet_manager.cc:1939] T 3b52c4dbcca14cb6aca4914e38f45035 P 4ae03fe546924cad854258eb297dc52a: Deleting consensus metadata
I20260926 15:03:25.081519 17305 catalog_manager.cc:4841] TS 4ae03fe546924cad854258eb297dc52a (127.9.123.66:34747): tablet 3b52c4dbcca14cb6aca4914e38f45035 (table test-table [id=a2599273b1d641e6b774666e52c78cc8]) successfully deleted
I20260926 15:03:25.082322 17855 ts_tablet_manager.cc:1905] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:03:25.082538 17596 tablet_service.cc:1467] Processing CreateTablet for tablet 0ba0a79ad7fb4e6a895cc46847525cb6 (DEFAULT_TABLE table=test-table [id=d98b0706549a4686af7766f0ee835384]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:03:25.082870 17856 ts_tablet_manager.cc:1905] T 3b52c4dbcca14cb6aca4914e38f45035 P e9d05cc314cb404086e36463718888a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:03:25.082958 17596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ba0a79ad7fb4e6a895cc46847525cb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:25.083793 17831 tablet_bootstrap.cc:492] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e9d05cc314cb404086e36463718888a5: Bootstrap starting.
I20260926 15:03:25.085450 17831 tablet_bootstrap.cc:654] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e9d05cc314cb404086e36463718888a5: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:25.089658 17855 ts_tablet_manager.cc:1918] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:03:25.089952 17855 log.cc:1196] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-0/wal/wals/3b52c4dbcca14cb6aca4914e38f45035
I20260926 15:03:25.090682 17855 ts_tablet_manager.cc:1939] T 3b52c4dbcca14cb6aca4914e38f45035 P e5a1a54cb63c445380be679d749f8c62: Deleting consensus metadata
I20260926 15:03:25.092744 17304 catalog_manager.cc:4841] TS e5a1a54cb63c445380be679d749f8c62 (127.9.123.65:33917): tablet 3b52c4dbcca14cb6aca4914e38f45035 (table test-table [id=a2599273b1d641e6b774666e52c78cc8]) successfully deleted
I20260926 15:03:25.093884 17856 ts_tablet_manager.cc:1918] T 3b52c4dbcca14cb6aca4914e38f45035 P e9d05cc314cb404086e36463718888a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:03:25.094180 17856 log.cc:1196] T 3b52c4dbcca14cb6aca4914e38f45035 P e9d05cc314cb404086e36463718888a5: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/3b52c4dbcca14cb6aca4914e38f45035
I20260926 15:03:25.094776 17856 ts_tablet_manager.cc:1939] T 3b52c4dbcca14cb6aca4914e38f45035 P e9d05cc314cb404086e36463718888a5: Deleting consensus metadata
I20260926 15:03:25.096633 17304 catalog_manager.cc:4841] TS e9d05cc314cb404086e36463718888a5 (127.9.123.67:42353): tablet 3b52c4dbcca14cb6aca4914e38f45035 (table test-table [id=a2599273b1d641e6b774666e52c78cc8]) successfully deleted
I20260926 15:03:25.097467 17832 tablet_bootstrap.cc:492] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62: Bootstrap starting.
I20260926 15:03:25.097538 17830 tablet_bootstrap.cc:492] T 0ba0a79ad7fb4e6a895cc46847525cb6 P 4ae03fe546924cad854258eb297dc52a: Bootstrap starting.
I20260926 15:03:25.098875 17830 tablet_bootstrap.cc:654] T 0ba0a79ad7fb4e6a895cc46847525cb6 P 4ae03fe546924cad854258eb297dc52a: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:25.099118 17832 tablet_bootstrap.cc:654] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:25.101140 17830 tablet_bootstrap.cc:492] T 0ba0a79ad7fb4e6a895cc46847525cb6 P 4ae03fe546924cad854258eb297dc52a: No bootstrap required, opened a new log
I20260926 15:03:25.101303 17830 ts_tablet_manager.cc:1397] T 0ba0a79ad7fb4e6a895cc46847525cb6 P 4ae03fe546924cad854258eb297dc52a: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260926 15:03:25.102542 17830 raft_consensus.cc:348] T 0ba0a79ad7fb4e6a895cc46847525cb6 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:25.102885 17832 tablet_bootstrap.cc:492] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62: No bootstrap required, opened a new log
I20260926 15:03:25.102916 17830 raft_consensus.cc:374] T 0ba0a79ad7fb4e6a895cc46847525cb6 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:25.103055 17832 ts_tablet_manager.cc:1397] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260926 15:03:25.103760 17831 tablet_bootstrap.cc:492] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e9d05cc314cb404086e36463718888a5: No bootstrap required, opened a new log
I20260926 15:03:25.103915 17831 ts_tablet_manager.cc:1397] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e9d05cc314cb404086e36463718888a5: Time spent bootstrapping tablet: real 0.020s user 0.009s sys 0.005s
I20260926 15:03:25.103902 17832 raft_consensus.cc:348] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:25.104135 17832 raft_consensus.cc:374] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:25.104161 17830 raft_consensus.cc:729] T 0ba0a79ad7fb4e6a895cc46847525cb6 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ae03fe546924cad854258eb297dc52a, State: Initialized, Role: FOLLOWER
I20260926 15:03:25.104240 17832 raft_consensus.cc:729] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5a1a54cb63c445380be679d749f8c62, State: Initialized, Role: FOLLOWER
I20260926 15:03:25.104475 17832 consensus_queue.cc:260] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:25.104475 17830 consensus_queue.cc:260] T 0ba0a79ad7fb4e6a895cc46847525cb6 P 4ae03fe546924cad854258eb297dc52a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:25.104688 17831 raft_consensus.cc:348] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:25.104911 17831 raft_consensus.cc:374] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:25.104995 17831 raft_consensus.cc:729] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d05cc314cb404086e36463718888a5, State: Initialized, Role: FOLLOWER
I20260926 15:03:25.105077 17832 ts_tablet_manager.cc:1428] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260926 15:03:25.105290 17831 consensus_queue.cc:260] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e9d05cc314cb404086e36463718888a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:25.105849 17831 ts_tablet_manager.cc:1428] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e9d05cc314cb404086e36463718888a5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260926 15:03:25.107491 17830 ts_tablet_manager.cc:1428] T 0ba0a79ad7fb4e6a895cc46847525cb6 P 4ae03fe546924cad854258eb297dc52a: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
W20260926 15:03:25.316818 17667 tablet.cc:2369] T 0ba0a79ad7fb4e6a895cc46847525cb6 P 4ae03fe546924cad854258eb297dc52a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 15:03:25.344907 17806 tablet.cc:2369] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e9d05cc314cb404086e36463718888a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:03:25.457388 17849 raft_consensus.cc:482] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:03:25.457628 17849 raft_consensus.cc:504] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:25.458540 17849 leader_election.cc:288] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ae03fe546924cad854258eb297dc52a (127.9.123.66:34747), e9d05cc314cb404086e36463718888a5 (127.9.123.67:42353)
I20260926 15:03:25.459049 17616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ba0a79ad7fb4e6a895cc46847525cb6" candidate_uuid: "e5a1a54cb63c445380be679d749f8c62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ae03fe546924cad854258eb297dc52a" is_pre_election: true
I20260926 15:03:25.459337 17616 raft_consensus.cc:2454] T 0ba0a79ad7fb4e6a895cc46847525cb6 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5a1a54cb63c445380be679d749f8c62 in term 0.
I20260926 15:03:25.459321 17755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ba0a79ad7fb4e6a895cc46847525cb6" candidate_uuid: "e5a1a54cb63c445380be679d749f8c62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d05cc314cb404086e36463718888a5" is_pre_election: true
I20260926 15:03:25.459566 17755 raft_consensus.cc:2454] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5a1a54cb63c445380be679d749f8c62 in term 0.
I20260926 15:03:25.459841 17408 leader_election.cc:302] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ae03fe546924cad854258eb297dc52a, e5a1a54cb63c445380be679d749f8c62; no voters:
I20260926 15:03:25.460124 17849 raft_consensus.cc:2781] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:03:25.460247 17849 raft_consensus.cc:482] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:03:25.460341 17849 raft_consensus.cc:3037] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:25.462519 17849 raft_consensus.cc:504] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:25.463070 17849 leader_election.cc:288] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62 [CANDIDATE]: Term 1 election: Requested vote from peers 4ae03fe546924cad854258eb297dc52a (127.9.123.66:34747), e9d05cc314cb404086e36463718888a5 (127.9.123.67:42353)
I20260926 15:03:25.463529 17616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ba0a79ad7fb4e6a895cc46847525cb6" candidate_uuid: "e5a1a54cb63c445380be679d749f8c62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ae03fe546924cad854258eb297dc52a"
I20260926 15:03:25.463622 17755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ba0a79ad7fb4e6a895cc46847525cb6" candidate_uuid: "e5a1a54cb63c445380be679d749f8c62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d05cc314cb404086e36463718888a5"
I20260926 15:03:25.463738 17616 raft_consensus.cc:3037] T 0ba0a79ad7fb4e6a895cc46847525cb6 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:25.463804 17755 raft_consensus.cc:3037] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:25.466562 17616 raft_consensus.cc:2454] T 0ba0a79ad7fb4e6a895cc46847525cb6 P 4ae03fe546924cad854258eb297dc52a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5a1a54cb63c445380be679d749f8c62 in term 1.
I20260926 15:03:25.466562 17755 raft_consensus.cc:2454] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e9d05cc314cb404086e36463718888a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5a1a54cb63c445380be679d749f8c62 in term 1.
I20260926 15:03:25.466944 17408 leader_election.cc:302] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ae03fe546924cad854258eb297dc52a, e5a1a54cb63c445380be679d749f8c62; no voters:
I20260926 15:03:25.467319 17849 raft_consensus.cc:2781] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:03:25.467444 17849 raft_consensus.cc:686] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62 [term 1 LEADER]: Becoming Leader. State: Replica: e5a1a54cb63c445380be679d749f8c62, State: Running, Role: LEADER
I20260926 15:03:25.467680 17849 consensus_queue.cc:237] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:25.470243 17320 catalog_manager.cc:5495] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62 reported cstate change: term changed from 0 to 1, leader changed from <none> to e5a1a54cb63c445380be679d749f8c62 (127.9.123.65). New cstate: current_term: 1 leader_uuid: "e5a1a54cb63c445380be679d749f8c62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } health_report { overall_health: UNKNOWN } } }
Sep 26 15:03:25 dist-test-slave-7153 krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435002, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.126@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM
I20260926 15:03:25.580778 17867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:25.567562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:41723 (local address 127.0.0.1:37596)
0926 15:03:25.568293 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:25.568333 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:25.568491 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:25.568859 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:25.568863 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:25.569562 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:25.569764 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:25.569773 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.570856 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.570860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:25.571994 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:25.572008 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.572747 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.572752 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:25.572869 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:25.573543 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:25.573565 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:25.575703 (+ 2138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:25.578670 (+ 2967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:25.578683 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:25.578689 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:25.579131 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:25.579432 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:25.579435 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:25.579437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:25.579565 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:25.580003 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:25.580009 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:25.580161 (+ 152us) client_negotiation.cc:769] Sending connection context
0926 15:03:25.580284 (+ 123us) client_negotiation.cc:240] Negotiation successful
0926 15:03:25.580503 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":485,"threads_started":1}
I20260926 15:03:25.580863 17866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:25.567523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37596 (local address 127.9.123.126:41723)
0926 15:03:25.568161 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:25.568170 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:25.568473 (+ 303us) server_negotiation.cc:407] Connection header received
0926 15:03:25.568604 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:25.568607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:25.568686 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:25.568779 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:25.569908 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.570700 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.572151 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.572582 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.572680 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:25.575899 (+ 3219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:25.575905 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:25.575911 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:25.575971 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:25.578536 (+ 2565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:25.579227 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:25.579231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:25.579233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:25.579347 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:25.579644 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:25.579646 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:25.579648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:25.579898 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:25.580012 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:25.580374 (+ 362us) server_negotiation.cc:299] Negotiation successful
0926 15:03:25.580603 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":471,"thread_start_us":298,"threads_started":1}
I20260926 15:03:25.585588 17320 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:37596:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260926 15:03:25.586531 17320 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:03:25.593274 17457 tablet_service.cc:1467] Processing CreateTablet for tablet 9b71c9d243584da4b41b0909d03c4369 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cc922d3083764d008131a3ca5a545366]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 15:03:25.593329 17596 tablet_service.cc:1467] Processing CreateTablet for tablet 9b71c9d243584da4b41b0909d03c4369 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cc922d3083764d008131a3ca5a545366]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 15:03:25.593504 17735 tablet_service.cc:1467] Processing CreateTablet for tablet 9b71c9d243584da4b41b0909d03c4369 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cc922d3083764d008131a3ca5a545366]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260926 15:03:25.593868 17457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b71c9d243584da4b41b0909d03c4369. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:25.593885 17596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b71c9d243584da4b41b0909d03c4369. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:25.593930 17735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b71c9d243584da4b41b0909d03c4369. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:25.599634 17830 tablet_bootstrap.cc:492] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a: Bootstrap starting.
I20260926 15:03:25.600458 17832 tablet_bootstrap.cc:492] T 9b71c9d243584da4b41b0909d03c4369 P e5a1a54cb63c445380be679d749f8c62: Bootstrap starting.
I20260926 15:03:25.600811 17831 tablet_bootstrap.cc:492] T 9b71c9d243584da4b41b0909d03c4369 P e9d05cc314cb404086e36463718888a5: Bootstrap starting.
I20260926 15:03:25.601260 17830 tablet_bootstrap.cc:654] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:25.601827 17832 tablet_bootstrap.cc:654] T 9b71c9d243584da4b41b0909d03c4369 P e5a1a54cb63c445380be679d749f8c62: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:25.602367 17831 tablet_bootstrap.cc:654] T 9b71c9d243584da4b41b0909d03c4369 P e9d05cc314cb404086e36463718888a5: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:25.603508 17832 tablet_bootstrap.cc:492] T 9b71c9d243584da4b41b0909d03c4369 P e5a1a54cb63c445380be679d749f8c62: No bootstrap required, opened a new log
I20260926 15:03:25.603636 17832 ts_tablet_manager.cc:1397] T 9b71c9d243584da4b41b0909d03c4369 P e5a1a54cb63c445380be679d749f8c62: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260926 15:03:25.604230 17832 raft_consensus.cc:348] T 9b71c9d243584da4b41b0909d03c4369 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:25.604463 17832 raft_consensus.cc:374] T 9b71c9d243584da4b41b0909d03c4369 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:25.604535 17832 raft_consensus.cc:729] T 9b71c9d243584da4b41b0909d03c4369 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5a1a54cb63c445380be679d749f8c62, State: Initialized, Role: FOLLOWER
I20260926 15:03:25.604697 17831 tablet_bootstrap.cc:492] T 9b71c9d243584da4b41b0909d03c4369 P e9d05cc314cb404086e36463718888a5: No bootstrap required, opened a new log
I20260926 15:03:25.604777 17832 consensus_queue.cc:260] T 9b71c9d243584da4b41b0909d03c4369 P e5a1a54cb63c445380be679d749f8c62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:25.605124 17831 ts_tablet_manager.cc:1397] T 9b71c9d243584da4b41b0909d03c4369 P e9d05cc314cb404086e36463718888a5: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260926 15:03:25.605218 17849 tablet_replica.cc:439] T 9b71c9d243584da4b41b0909d03c4369 P e5a1a54cb63c445380be679d749f8c62: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } } }
I20260926 15:03:25.605473 17849 tablet_replica.cc:442] T 9b71c9d243584da4b41b0909d03c4369 P e5a1a54cb63c445380be679d749f8c62: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 15:03:25.605952 17830 tablet_bootstrap.cc:492] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a: No bootstrap required, opened a new log
I20260926 15:03:25.605849 17831 raft_consensus.cc:348] T 9b71c9d243584da4b41b0909d03c4369 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:25.606065 17831 raft_consensus.cc:374] T 9b71c9d243584da4b41b0909d03c4369 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:25.606113 17830 ts_tablet_manager.cc:1397] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.005s
I20260926 15:03:25.606143 17831 raft_consensus.cc:729] T 9b71c9d243584da4b41b0909d03c4369 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d05cc314cb404086e36463718888a5, State: Initialized, Role: FOLLOWER
I20260926 15:03:25.606360 17831 consensus_queue.cc:260] T 9b71c9d243584da4b41b0909d03c4369 P e9d05cc314cb404086e36463718888a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:25.606628 17832 ts_tablet_manager.cc:1428] T 9b71c9d243584da4b41b0909d03c4369 P e5a1a54cb63c445380be679d749f8c62: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260926 15:03:25.607097 17831 ts_tablet_manager.cc:1428] T 9b71c9d243584da4b41b0909d03c4369 P e9d05cc314cb404086e36463718888a5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260926 15:03:25.607071 17871 tablet_replica.cc:439] T 9b71c9d243584da4b41b0909d03c4369 P e9d05cc314cb404086e36463718888a5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } } }
I20260926 15:03:25.607515 17871 tablet_replica.cc:442] T 9b71c9d243584da4b41b0909d03c4369 P e9d05cc314cb404086e36463718888a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 15:03:25.607967 17830 raft_consensus.cc:348] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:25.608197 17830 raft_consensus.cc:374] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:25.608282 17830 raft_consensus.cc:729] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ae03fe546924cad854258eb297dc52a, State: Initialized, Role: FOLLOWER
I20260926 15:03:25.608529 17830 consensus_queue.cc:260] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:25.609263 17830 ts_tablet_manager.cc:1428] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260926 15:03:25.609831 17872 tablet_replica.cc:439] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } } }
I20260926 15:03:25.610075 17872 tablet_replica.cc:442] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 15:03:25.658215 17872 raft_consensus.cc:482] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:03:25.658435 17872 raft_consensus.cc:504] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:25.659999 17872 leader_election.cc:288] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5a1a54cb63c445380be679d749f8c62 (127.9.123.65:33917), e9d05cc314cb404086e36463718888a5 (127.9.123.67:42353)
I20260926 15:03:25.666780 17873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:25.660252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:33917 (local address 127.9.123.66:33479)
0926 15:03:25.660812 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:25.660839 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:25.660982 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:25.661396 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:25.661400 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:25.661441 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:25.661838 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:25.661850 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.663200 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.663203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:25.664968 (+ 1765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:25.664982 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.666236 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.666241 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:25.666348 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:25.666373 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 15:03:25.666466 (+ 93us) client_negotiation.cc:240] Negotiation successful
0926 15:03:25.666560 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":274,"threads_started":1}
I20260926 15:03:25.666935 17874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:25.660442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:33479 (local address 127.9.123.65:33917)
0926 15:03:25.661135 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:25.661141 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:25.661159 (+ 18us) server_negotiation.cc:407] Connection header received
0926 15:03:25.661213 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:25.661217 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:25.661274 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:25.661355 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:25.662057 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.663077 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.665172 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.666092 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.666263 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:25.666356 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:25.666500 (+ 144us) server_negotiation.cc:299] Negotiation successful
0926 15:03:25.666627 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":403,"threads_started":1}
I20260926 15:03:25.667861 17875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:25.661022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:42353 (local address 127.9.123.66:60341)
0926 15:03:25.661957 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:25.661974 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:25.662063 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:25.662462 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:25.662465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:25.662482 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:25.662736 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:25.662743 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.663960 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.663965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:25.665667 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:25.665678 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.667180 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.667186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:25.667359 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:25.667389 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 15:03:25.667528 (+ 139us) client_negotiation.cc:240] Negotiation successful
0926 15:03:25.667642 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"thread_start_us":579,"threads_started":1}
I20260926 15:03:25.667754 17481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b71c9d243584da4b41b0909d03c4369" candidate_uuid: "4ae03fe546924cad854258eb297dc52a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5a1a54cb63c445380be679d749f8c62" is_pre_election: true
I20260926 15:03:25.667953 17876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:25.661186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:60341 (local address 127.9.123.67:42353)
0926 15:03:25.662176 (+ 990us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:25.662182 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:25.662201 (+ 19us) server_negotiation.cc:407] Connection header received
0926 15:03:25.662266 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:25.662271 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:25.662340 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:25.662450 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:25.662884 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.663806 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.665843 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.666990 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.667149 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:25.667213 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:25.667563 (+ 350us) server_negotiation.cc:299] Negotiation successful
0926 15:03:25.667694 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":865,"thread_start_us":706,"threads_started":1}
I20260926 15:03:25.668159 17481 raft_consensus.cc:2454] T 9b71c9d243584da4b41b0909d03c4369 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ae03fe546924cad854258eb297dc52a in term 0.
I20260926 15:03:25.668444 17755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b71c9d243584da4b41b0909d03c4369" candidate_uuid: "4ae03fe546924cad854258eb297dc52a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d05cc314cb404086e36463718888a5" is_pre_election: true
I20260926 15:03:25.668675 17755 raft_consensus.cc:2454] T 9b71c9d243584da4b41b0909d03c4369 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ae03fe546924cad854258eb297dc52a in term 0.
I20260926 15:03:25.668733 17547 leader_election.cc:302] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ae03fe546924cad854258eb297dc52a, e5a1a54cb63c445380be679d749f8c62; no voters:
I20260926 15:03:25.669162 17872 raft_consensus.cc:2781] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:03:25.669279 17872 raft_consensus.cc:482] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:03:25.669378 17872 raft_consensus.cc:3037] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:25.671396 17872 raft_consensus.cc:504] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:25.671959 17872 leader_election.cc:288] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a [CANDIDATE]: Term 1 election: Requested vote from peers e5a1a54cb63c445380be679d749f8c62 (127.9.123.65:33917), e9d05cc314cb404086e36463718888a5 (127.9.123.67:42353)
I20260926 15:03:25.672521 17481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b71c9d243584da4b41b0909d03c4369" candidate_uuid: "4ae03fe546924cad854258eb297dc52a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5a1a54cb63c445380be679d749f8c62"
I20260926 15:03:25.672652 17755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b71c9d243584da4b41b0909d03c4369" candidate_uuid: "4ae03fe546924cad854258eb297dc52a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d05cc314cb404086e36463718888a5"
I20260926 15:03:25.672726 17481 raft_consensus.cc:3037] T 9b71c9d243584da4b41b0909d03c4369 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:25.672868 17755 raft_consensus.cc:3037] T 9b71c9d243584da4b41b0909d03c4369 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:25.674784 17755 raft_consensus.cc:2454] T 9b71c9d243584da4b41b0909d03c4369 P e9d05cc314cb404086e36463718888a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ae03fe546924cad854258eb297dc52a in term 1.
I20260926 15:03:25.674810 17481 raft_consensus.cc:2454] T 9b71c9d243584da4b41b0909d03c4369 P e5a1a54cb63c445380be679d749f8c62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ae03fe546924cad854258eb297dc52a in term 1.
I20260926 15:03:25.675159 17547 leader_election.cc:302] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ae03fe546924cad854258eb297dc52a, e9d05cc314cb404086e36463718888a5; no voters:
I20260926 15:03:25.675474 17872 raft_consensus.cc:2781] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:03:25.675907 17872 raft_consensus.cc:686] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a [term 1 LEADER]: Becoming Leader. State: Replica: 4ae03fe546924cad854258eb297dc52a, State: Running, Role: LEADER
I20260926 15:03:25.676330 17872 consensus_queue.cc:237] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:25.678193 17877 tablet_replica.cc:439] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a: TxnStatusTablet state changed. Reason: New leader 4ae03fe546924cad854258eb297dc52a. Latest consensus state: current_term: 1 leader_uuid: "4ae03fe546924cad854258eb297dc52a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } } }
I20260926 15:03:25.678617 17877 tablet_replica.cc:442] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a: This TxnStatusTablet replica's current role is: LEADER
I20260926 15:03:25.679234 17879 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260926 15:03:25.679417 17879 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260926 15:03:25.679641 17319 catalog_manager.cc:5495] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ae03fe546924cad854258eb297dc52a (127.9.123.66). New cstate: current_term: 1 leader_uuid: "4ae03fe546924cad854258eb297dc52a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } health_report { overall_health: UNKNOWN } } }
I20260926 15:03:25.710626 17882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:25.704175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55712 (local address 127.9.123.66:34747)
0926 15:03:25.704801 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:25.704806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:25.704825 (+ 19us) server_negotiation.cc:407] Connection header received
0926 15:03:25.704903 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:25.704906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:25.704997 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:25.705161 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:03:25.705573 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.706375 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.708229 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.708684 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.708777 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:25.709352 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:03:25.709999 (+ 647us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:03:25.710116 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:25.710257 (+ 141us) server_negotiation.cc:299] Negotiation successful
0926 15:03:25.710330 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":367,"threads_started":1}
I20260926 15:03:25.710722 17867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:25.704027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:34747 (local address 127.0.0.1:55712)
0926 15:03:25.704297 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:25.704318 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:25.704488 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:25.705195 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:25.705201 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:25.705214 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 15:03:25.705419 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:25.705431 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.706499 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.706506 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:25.708085 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:25.708100 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.708836 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.708843 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:25.709032 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:25.709144 (+ 112us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 15:03:25.710112 (+ 968us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 15:03:25.710136 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 15:03:25.710227 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 15:03:25.710405 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260926 15:03:25.715314 17755 raft_consensus.cc:1261] T 9b71c9d243584da4b41b0909d03c4369 P e9d05cc314cb404086e36463718888a5 [term 1 FOLLOWER]: Refusing update from remote peer 4ae03fe546924cad854258eb297dc52a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:03:25.715417 17481 raft_consensus.cc:1261] T 9b71c9d243584da4b41b0909d03c4369 P e5a1a54cb63c445380be679d749f8c62 [term 1 FOLLOWER]: Refusing update from remote peer 4ae03fe546924cad854258eb297dc52a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:03:25.716125 17872 consensus_queue.cc:1035] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:03:25.716475 17877 consensus_queue.cc:1035] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:03:25.722004 17871 tablet_replica.cc:439] T 9b71c9d243584da4b41b0909d03c4369 P e9d05cc314cb404086e36463718888a5: TxnStatusTablet state changed. Reason: New leader 4ae03fe546924cad854258eb297dc52a. Latest consensus state: current_term: 1 leader_uuid: "4ae03fe546924cad854258eb297dc52a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } } }
I20260926 15:03:25.722296 17871 tablet_replica.cc:442] T 9b71c9d243584da4b41b0909d03c4369 P e9d05cc314cb404086e36463718888a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 15:03:25.724478 17883 mvcc.cc:205] Tried to move back new op lower bound from 7333621783401959424 to 7333621783254032384. Current Snapshot: MvccSnapshot[applied={T|T < 7333621783401959424}]
I20260926 15:03:25.734876 17871 tablet_replica.cc:439] T 9b71c9d243584da4b41b0909d03c4369 P e9d05cc314cb404086e36463718888a5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ae03fe546924cad854258eb297dc52a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } } }
I20260926 15:03:25.735215 17871 tablet_replica.cc:442] T 9b71c9d243584da4b41b0909d03c4369 P e9d05cc314cb404086e36463718888a5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 15:03:25.736642 17849 tablet_replica.cc:439] T 9b71c9d243584da4b41b0909d03c4369 P e5a1a54cb63c445380be679d749f8c62: TxnStatusTablet state changed. Reason: New leader 4ae03fe546924cad854258eb297dc52a. Latest consensus state: current_term: 1 leader_uuid: "4ae03fe546924cad854258eb297dc52a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } } }
I20260926 15:03:25.736964 17849 tablet_replica.cc:442] T 9b71c9d243584da4b41b0909d03c4369 P e5a1a54cb63c445380be679d749f8c62: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 15:03:25.737757 17872 tablet_replica.cc:439] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ae03fe546924cad854258eb297dc52a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } } }
I20260926 15:03:25.738034 17872 tablet_replica.cc:442] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a: This TxnStatusTablet replica's current role is: LEADER
I20260926 15:03:25.744067 17872 tablet_replica.cc:439] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ae03fe546924cad854258eb297dc52a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } } }
I20260926 15:03:25.744347 17872 tablet_replica.cc:442] T 9b71c9d243584da4b41b0909d03c4369 P 4ae03fe546924cad854258eb297dc52a: This TxnStatusTablet replica's current role is: LEADER
I20260926 15:03:25.750144 17849 tablet_replica.cc:439] T 9b71c9d243584da4b41b0909d03c4369 P e5a1a54cb63c445380be679d749f8c62: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ae03fe546924cad854258eb297dc52a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } } }
I20260926 15:03:25.750490 17849 tablet_replica.cc:442] T 9b71c9d243584da4b41b0909d03c4369 P e5a1a54cb63c445380be679d749f8c62: This TxnStatusTablet replica's current role is: FOLLOWER
I20260926 15:03:25.760313 17882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:25.754003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:54771 (local address 127.9.123.66:34747)
0926 15:03:25.754232 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:25.754239 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:25.754845 (+ 606us) server_negotiation.cc:407] Connection header received
0926 15:03:25.755031 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:25.755035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:25.755100 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:25.755175 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:03:25.755695 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.756472 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.758493 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.758927 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.759056 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:25.759629 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:03:25.759748 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:03:25.759819 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:25.759981 (+ 162us) server_negotiation.cc:299] Negotiation successful
0926 15:03:25.760049 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260926 15:03:25.760517 17894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:25.753897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:34747 (local address 127.9.123.65:54771)
0926 15:03:25.754728 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:25.754749 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:25.754906 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:25.755209 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:25.755214 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:25.755227 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 15:03:25.755548 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:25.755561 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.756603 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.756609 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:25.758316 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:25.758332 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.759105 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.759112 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:25.759308 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:25.759392 (+ 84us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 15:03:25.759853 (+ 461us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 15:03:25.759865 (+ 12us) client_negotiation.cc:769] Sending connection context
0926 15:03:25.759951 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 15:03:25.760120 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":504,"threads_started":1}
I20260926 15:03:25.770587 17755 raft_consensus.cc:1261] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e9d05cc314cb404086e36463718888a5 [term 1 FOLLOWER]: Refusing update from remote peer e5a1a54cb63c445380be679d749f8c62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:03:25.770642 17616 raft_consensus.cc:1261] T 0ba0a79ad7fb4e6a895cc46847525cb6 P 4ae03fe546924cad854258eb297dc52a [term 1 FOLLOWER]: Refusing update from remote peer e5a1a54cb63c445380be679d749f8c62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:03:25.771210 17849 consensus_queue.cc:1035] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:03:25.771508 17837 consensus_queue.cc:1035] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:03:25.827970 17874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:25.820717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:42637 (local address 127.9.123.65:33917)
0926 15:03:25.820903 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:25.820909 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:25.821060 (+ 151us) server_negotiation.cc:407] Connection header received
0926 15:03:25.821366 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:25.821371 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:25.821464 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:25.821560 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:03:25.822194 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.823291 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.825716 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.826428 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.826550 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:25.827131 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:03:25.827292 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:03:25.827358 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:25.827594 (+ 236us) server_negotiation.cc:299] Negotiation successful
0926 15:03:25.827688 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 15:03:25.828826 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:25.819961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:33917 (local address 127.9.123.66:42637)
0926 15:03:25.820969 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:25.820991 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:25.821200 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:25.821587 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:25.821592 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:25.821605 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0926 15:03:25.822014 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:25.822025 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.823486 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.823491 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:25.825566 (+ 2075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:25.825581 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.826621 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.826628 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:25.826816 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:25.826894 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0926 15:03:25.827440 (+ 546us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0926 15:03:25.827450 (+ 10us) client_negotiation.cc:769] Sending connection context
0926 15:03:25.828239 (+ 789us) client_negotiation.cc:240] Negotiation successful
0926 15:03:25.828402 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":797,"threads_started":1}
I20260926 15:03:25.872558 17320 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:03:25.872776 17320 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:03:25.875607 17320 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07: Sending DeleteTablet for 3 replicas of tablet 0ba0a79ad7fb4e6a895cc46847525cb6
I20260926 15:03:25.876379 17457 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ba0a79ad7fb4e6a895cc46847525cb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:03:25 UTC) from {username='oryx', principal='oryx/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:37844
I20260926 15:03:25.876621 17596 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ba0a79ad7fb4e6a895cc46847525cb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:03:25 UTC) from {username='oryx', principal='oryx/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:55700
I20260926 15:03:25.877321 17735 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ba0a79ad7fb4e6a895cc46847525cb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:03:25 UTC) from {username='oryx', principal='oryx/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:60510
I20260926 15:03:25.877604 17902 tablet_replica.cc:330] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62: stopping tablet replica
I20260926 15:03:25.877810 17902 raft_consensus.cc:2229] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62 [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:03:25.878196 17902 raft_consensus.cc:2258] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:25.879220 17902 ts_tablet_manager.cc:1905] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:03:25.886665 17902 ts_tablet_manager.cc:1918] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 15:03:25.886840 17902 log.cc:1196] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-0/wal/wals/0ba0a79ad7fb4e6a895cc46847525cb6
I20260926 15:03:25.887063 17903 tablet_replica.cc:330] T 0ba0a79ad7fb4e6a895cc46847525cb6 P 4ae03fe546924cad854258eb297dc52a: stopping tablet replica
I20260926 15:03:25.887354 17902 ts_tablet_manager.cc:1939] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e5a1a54cb63c445380be679d749f8c62: Deleting consensus metadata
I20260926 15:03:25.887352 17903 raft_consensus.cc:2229] T 0ba0a79ad7fb4e6a895cc46847525cb6 P 4ae03fe546924cad854258eb297dc52a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:03:25.887804 17903 raft_consensus.cc:2258] T 0ba0a79ad7fb4e6a895cc46847525cb6 P 4ae03fe546924cad854258eb297dc52a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:25.888536 17304 catalog_manager.cc:4841] TS e5a1a54cb63c445380be679d749f8c62 (127.9.123.65:33917): tablet 0ba0a79ad7fb4e6a895cc46847525cb6 (table test-table [id=d98b0706549a4686af7766f0ee835384]) successfully deleted
I20260926 15:03:25.889173 17903 ts_tablet_manager.cc:1905] T 0ba0a79ad7fb4e6a895cc46847525cb6 P 4ae03fe546924cad854258eb297dc52a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:03:25.897827 17903 ts_tablet_manager.cc:1918] T 0ba0a79ad7fb4e6a895cc46847525cb6 P 4ae03fe546924cad854258eb297dc52a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 15:03:25.898063 17904 tablet_replica.cc:330] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e9d05cc314cb404086e36463718888a5: stopping tablet replica
I20260926 15:03:25.898133 17903 log.cc:1196] T 0ba0a79ad7fb4e6a895cc46847525cb6 P 4ae03fe546924cad854258eb297dc52a: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/0ba0a79ad7fb4e6a895cc46847525cb6
I20260926 15:03:25.898291 17904 raft_consensus.cc:2229] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e9d05cc314cb404086e36463718888a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:03:25.898623 17904 raft_consensus.cc:2258] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e9d05cc314cb404086e36463718888a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:25.898826 17903 ts_tablet_manager.cc:1939] T 0ba0a79ad7fb4e6a895cc46847525cb6 P 4ae03fe546924cad854258eb297dc52a: Deleting consensus metadata
I20260926 15:03:25.899871 17904 ts_tablet_manager.cc:1905] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e9d05cc314cb404086e36463718888a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:03:25.900044 17305 catalog_manager.cc:4841] TS 4ae03fe546924cad854258eb297dc52a (127.9.123.66:34747): tablet 0ba0a79ad7fb4e6a895cc46847525cb6 (table test-table [id=d98b0706549a4686af7766f0ee835384]) successfully deleted
I20260926 15:03:25.907471 17904 ts_tablet_manager.cc:1918] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e9d05cc314cb404086e36463718888a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 15:03:25.907627 17904 log.cc:1196] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e9d05cc314cb404086e36463718888a5: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/0ba0a79ad7fb4e6a895cc46847525cb6
I20260926 15:03:25.908070 17904 ts_tablet_manager.cc:1939] T 0ba0a79ad7fb4e6a895cc46847525cb6 P e9d05cc314cb404086e36463718888a5: Deleting consensus metadata
I20260926 15:03:25.909392 17304 catalog_manager.cc:4841] TS e9d05cc314cb404086e36463718888a5 (127.9.123.67:42353): tablet 0ba0a79ad7fb4e6a895cc46847525cb6 (table test-table [id=d98b0706549a4686af7766f0ee835384]) successfully deleted
Sep 26 15:03:25 dist-test-slave-7153 krb5kdc[17275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 26 15:03:25 dist-test-slave-7153 krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:25 dist-test-slave-7153 krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
W20260926 15:03:25.939249 17866 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:25.927965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37600 (local address 127.9.123.126:41723)
0926 15:03:25.928212 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:25.928219 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:25.928724 (+ 505us) server_negotiation.cc:407] Connection header received
0926 15:03:25.928904 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:25.928910 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:25.928976 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:25.929059 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:25.930482 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.931215 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.932419 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.932851 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.932971 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:25.938957 (+ 5986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37600: BlockingRecv error: recv got EOF from 127.0.0.1:37600 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Sep 26 15:03:25 dist-test-slave-7153 krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM
I20260926 15:03:25.957692 17866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:25.942067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37608 (local address 127.9.123.126:41723)
0926 15:03:25.942264 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:25.942269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:25.942539 (+ 270us) server_negotiation.cc:407] Connection header received
0926 15:03:25.942697 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:25.942701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:25.942765 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:25.942859 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:25.944473 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.945268 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.947030 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:25.947453 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:25.947566 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:25.952382 (+ 4816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:25.952388 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:25.952391 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:25.952433 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:25.954992 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:25.955911 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:25.955914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:25.955916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:25.956072 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:25.956477 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:25.956480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:25.956482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:25.956723 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:25.956910 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:25.957260 (+ 350us) server_negotiation.cc:299] Negotiation successful
0926 15:03:25.957443 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 15:03:25.961839 17320 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37608:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:03:25.962605 17320 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:03:25.969709 17457 tablet_service.cc:1467] Processing CreateTablet for tablet 668f5a620b0c43da9d16d9ecb463caa8 (DEFAULT_TABLE table=test-table [id=49f77fabef0c4aa49d7ecf4493c2a927]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:03:25.969900 17596 tablet_service.cc:1467] Processing CreateTablet for tablet 668f5a620b0c43da9d16d9ecb463caa8 (DEFAULT_TABLE table=test-table [id=49f77fabef0c4aa49d7ecf4493c2a927]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:03:25.970079 17735 tablet_service.cc:1467] Processing CreateTablet for tablet 668f5a620b0c43da9d16d9ecb463caa8 (DEFAULT_TABLE table=test-table [id=49f77fabef0c4aa49d7ecf4493c2a927]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:03:25.970188 17457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 668f5a620b0c43da9d16d9ecb463caa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:25.970311 17596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 668f5a620b0c43da9d16d9ecb463caa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:25.970522 17735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 668f5a620b0c43da9d16d9ecb463caa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:25.976815 17832 tablet_bootstrap.cc:492] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62: Bootstrap starting.
I20260926 15:03:25.976840 17831 tablet_bootstrap.cc:492] T 668f5a620b0c43da9d16d9ecb463caa8 P e9d05cc314cb404086e36463718888a5: Bootstrap starting.
I20260926 15:03:25.978435 17832 tablet_bootstrap.cc:654] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:25.978448 17831 tablet_bootstrap.cc:654] T 668f5a620b0c43da9d16d9ecb463caa8 P e9d05cc314cb404086e36463718888a5: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:25.979843 17830 tablet_bootstrap.cc:492] T 668f5a620b0c43da9d16d9ecb463caa8 P 4ae03fe546924cad854258eb297dc52a: Bootstrap starting.
I20260926 15:03:25.981586 17830 tablet_bootstrap.cc:654] T 668f5a620b0c43da9d16d9ecb463caa8 P 4ae03fe546924cad854258eb297dc52a: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:25.989141 17831 tablet_bootstrap.cc:492] T 668f5a620b0c43da9d16d9ecb463caa8 P e9d05cc314cb404086e36463718888a5: No bootstrap required, opened a new log
I20260926 15:03:25.989320 17831 ts_tablet_manager.cc:1397] T 668f5a620b0c43da9d16d9ecb463caa8 P e9d05cc314cb404086e36463718888a5: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.004s
I20260926 15:03:25.989532 17832 tablet_bootstrap.cc:492] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62: No bootstrap required, opened a new log
I20260926 15:03:25.989708 17832 ts_tablet_manager.cc:1397] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.006s
I20260926 15:03:25.990103 17831 raft_consensus.cc:348] T 668f5a620b0c43da9d16d9ecb463caa8 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:25.990355 17831 raft_consensus.cc:374] T 668f5a620b0c43da9d16d9ecb463caa8 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:25.990577 17832 raft_consensus.cc:348] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:25.990840 17832 raft_consensus.cc:374] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:25.990938 17832 raft_consensus.cc:729] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5a1a54cb63c445380be679d749f8c62, State: Initialized, Role: FOLLOWER
I20260926 15:03:25.991214 17832 consensus_queue.cc:260] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:25.991582 17831 raft_consensus.cc:729] T 668f5a620b0c43da9d16d9ecb463caa8 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d05cc314cb404086e36463718888a5, State: Initialized, Role: FOLLOWER
I20260926 15:03:25.991870 17831 consensus_queue.cc:260] T 668f5a620b0c43da9d16d9ecb463caa8 P e9d05cc314cb404086e36463718888a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:25.992532 17831 ts_tablet_manager.cc:1428] T 668f5a620b0c43da9d16d9ecb463caa8 P e9d05cc314cb404086e36463718888a5: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260926 15:03:25.997501 17832 ts_tablet_manager.cc:1428] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20260926 15:03:25.999433 17830 tablet_bootstrap.cc:492] T 668f5a620b0c43da9d16d9ecb463caa8 P 4ae03fe546924cad854258eb297dc52a: No bootstrap required, opened a new log
I20260926 15:03:25.999620 17830 ts_tablet_manager.cc:1397] T 668f5a620b0c43da9d16d9ecb463caa8 P 4ae03fe546924cad854258eb297dc52a: Time spent bootstrapping tablet: real 0.020s user 0.012s sys 0.006s
I20260926 15:03:26.000502 17830 raft_consensus.cc:348] T 668f5a620b0c43da9d16d9ecb463caa8 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:26.000790 17830 raft_consensus.cc:374] T 668f5a620b0c43da9d16d9ecb463caa8 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:26.000887 17830 raft_consensus.cc:729] T 668f5a620b0c43da9d16d9ecb463caa8 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ae03fe546924cad854258eb297dc52a, State: Initialized, Role: FOLLOWER
I20260926 15:03:26.001166 17830 consensus_queue.cc:260] T 668f5a620b0c43da9d16d9ecb463caa8 P 4ae03fe546924cad854258eb297dc52a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:26.001713 17830 ts_tablet_manager.cc:1428] T 668f5a620b0c43da9d16d9ecb463caa8 P 4ae03fe546924cad854258eb297dc52a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260926 15:03:26.111835 17849 raft_consensus.cc:482] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:03:26.112015 17849 raft_consensus.cc:504] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:26.112686 17849 leader_election.cc:288] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ae03fe546924cad854258eb297dc52a (127.9.123.66:34747), e9d05cc314cb404086e36463718888a5 (127.9.123.67:42353)
I20260926 15:03:26.113336 17616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668f5a620b0c43da9d16d9ecb463caa8" candidate_uuid: "e5a1a54cb63c445380be679d749f8c62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ae03fe546924cad854258eb297dc52a" is_pre_election: true
I20260926 15:03:26.113336 17755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668f5a620b0c43da9d16d9ecb463caa8" candidate_uuid: "e5a1a54cb63c445380be679d749f8c62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d05cc314cb404086e36463718888a5" is_pre_election: true
I20260926 15:03:26.113580 17755 raft_consensus.cc:2454] T 668f5a620b0c43da9d16d9ecb463caa8 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5a1a54cb63c445380be679d749f8c62 in term 0.
I20260926 15:03:26.113590 17616 raft_consensus.cc:2454] T 668f5a620b0c43da9d16d9ecb463caa8 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5a1a54cb63c445380be679d749f8c62 in term 0.
I20260926 15:03:26.114087 17408 leader_election.cc:302] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4ae03fe546924cad854258eb297dc52a, e5a1a54cb63c445380be679d749f8c62, e9d05cc314cb404086e36463718888a5; no voters:
I20260926 15:03:26.114369 17849 raft_consensus.cc:2781] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:03:26.114495 17849 raft_consensus.cc:482] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:03:26.114584 17849 raft_consensus.cc:3037] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:26.117504 17849 raft_consensus.cc:504] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:26.118155 17849 leader_election.cc:288] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62 [CANDIDATE]: Term 1 election: Requested vote from peers 4ae03fe546924cad854258eb297dc52a (127.9.123.66:34747), e9d05cc314cb404086e36463718888a5 (127.9.123.67:42353)
I20260926 15:03:26.118714 17755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668f5a620b0c43da9d16d9ecb463caa8" candidate_uuid: "e5a1a54cb63c445380be679d749f8c62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d05cc314cb404086e36463718888a5"
I20260926 15:03:26.118736 17616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668f5a620b0c43da9d16d9ecb463caa8" candidate_uuid: "e5a1a54cb63c445380be679d749f8c62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ae03fe546924cad854258eb297dc52a"
I20260926 15:03:26.118882 17755 raft_consensus.cc:3037] T 668f5a620b0c43da9d16d9ecb463caa8 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:26.118933 17616 raft_consensus.cc:3037] T 668f5a620b0c43da9d16d9ecb463caa8 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:26.121021 17755 raft_consensus.cc:2454] T 668f5a620b0c43da9d16d9ecb463caa8 P e9d05cc314cb404086e36463718888a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5a1a54cb63c445380be679d749f8c62 in term 1.
I20260926 15:03:26.121516 17407 leader_election.cc:302] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5a1a54cb63c445380be679d749f8c62, e9d05cc314cb404086e36463718888a5; no voters:
I20260926 15:03:26.121793 17849 raft_consensus.cc:2781] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:03:26.121860 17616 raft_consensus.cc:2454] T 668f5a620b0c43da9d16d9ecb463caa8 P 4ae03fe546924cad854258eb297dc52a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5a1a54cb63c445380be679d749f8c62 in term 1.
I20260926 15:03:26.122025 17849 raft_consensus.cc:686] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62 [term 1 LEADER]: Becoming Leader. State: Replica: e5a1a54cb63c445380be679d749f8c62, State: Running, Role: LEADER
I20260926 15:03:26.122311 17849 consensus_queue.cc:237] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:26.124939 17319 catalog_manager.cc:5495] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62 reported cstate change: term changed from 0 to 1, leader changed from <none> to e5a1a54cb63c445380be679d749f8c62 (127.9.123.65). New cstate: current_term: 1 leader_uuid: "e5a1a54cb63c445380be679d749f8c62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } health_report { overall_health: UNKNOWN } } }
I20260926 15:03:26.148011 17874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:26.143514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37868 (local address 127.9.123.65:33917)
0926 15:03:26.143666 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:26.143670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:26.143767 (+ 97us) server_negotiation.cc:407] Connection header received
0926 15:03:26.143939 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:26.143951 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:26.144019 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:26.144101 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:03:26.144411 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:26.145243 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:26.146538 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:26.146935 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:26.147033 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:26.147445 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:03:26.147545 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:03:26.147605 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:26.147694 (+ 89us) server_negotiation.cc:299] Negotiation successful
0926 15:03:26.147763 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 15:03:26.150753 17755 raft_consensus.cc:1261] T 668f5a620b0c43da9d16d9ecb463caa8 P e9d05cc314cb404086e36463718888a5 [term 1 FOLLOWER]: Refusing update from remote peer e5a1a54cb63c445380be679d749f8c62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:03:26.150781 17616 raft_consensus.cc:1261] T 668f5a620b0c43da9d16d9ecb463caa8 P 4ae03fe546924cad854258eb297dc52a [term 1 FOLLOWER]: Refusing update from remote peer e5a1a54cb63c445380be679d749f8c62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:03:26.151419 17901 consensus_queue.cc:1035] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:03:26.151742 17849 consensus_queue.cc:1035] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:03:26.156978 17885 mvcc.cc:205] Tried to move back new op lower bound from 7333621785187713024 to 7333621785078894592. Current Snapshot: MvccSnapshot[applied={T|T < 7333621785187713024 or (T in {7333621785187713024})}]
I20260926 15:03:26.164642 17318 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:03:26.165199 17318 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:03:26.168358 17318 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07: Sending DeleteTablet for 3 replicas of tablet 668f5a620b0c43da9d16d9ecb463caa8
I20260926 15:03:26.169528 17457 tablet_service.cc:1514] Processing DeleteTablet for tablet 668f5a620b0c43da9d16d9ecb463caa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:03:26 UTC) from {username='oryx', principal='oryx/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:37844
I20260926 15:03:26.169351 17596 tablet_service.cc:1514] Processing DeleteTablet for tablet 668f5a620b0c43da9d16d9ecb463caa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:03:26 UTC) from {username='oryx', principal='oryx/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:55700
I20260926 15:03:26.169621 17735 tablet_service.cc:1514] Processing DeleteTablet for tablet 668f5a620b0c43da9d16d9ecb463caa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:03:26 UTC) from {username='oryx', principal='oryx/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:60510
I20260926 15:03:26.169860 17902 tablet_replica.cc:330] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62: stopping tablet replica
I20260926 15:03:26.170053 17902 raft_consensus.cc:2229] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62 [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:03:26.170342 17902 raft_consensus.cc:2258] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:26.170607 17903 tablet_replica.cc:330] T 668f5a620b0c43da9d16d9ecb463caa8 P 4ae03fe546924cad854258eb297dc52a: stopping tablet replica
I20260926 15:03:26.170795 17903 raft_consensus.cc:2229] T 668f5a620b0c43da9d16d9ecb463caa8 P 4ae03fe546924cad854258eb297dc52a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:03:26.170992 17903 raft_consensus.cc:2258] T 668f5a620b0c43da9d16d9ecb463caa8 P 4ae03fe546924cad854258eb297dc52a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:26.171247 17904 tablet_replica.cc:330] T 668f5a620b0c43da9d16d9ecb463caa8 P e9d05cc314cb404086e36463718888a5: stopping tablet replica
I20260926 15:03:26.171264 17902 ts_tablet_manager.cc:1905] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:03:26.171437 17904 raft_consensus.cc:2229] T 668f5a620b0c43da9d16d9ecb463caa8 P e9d05cc314cb404086e36463718888a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:03:26.171636 17904 raft_consensus.cc:2258] T 668f5a620b0c43da9d16d9ecb463caa8 P e9d05cc314cb404086e36463718888a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:26.173388 17903 ts_tablet_manager.cc:1905] T 668f5a620b0c43da9d16d9ecb463caa8 P 4ae03fe546924cad854258eb297dc52a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:03:26.173990 17318 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37608:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:03:26.174696 17318 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:03:26.176608 17904 ts_tablet_manager.cc:1905] T 668f5a620b0c43da9d16d9ecb463caa8 P e9d05cc314cb404086e36463718888a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:03:26.180013 17903 ts_tablet_manager.cc:1918] T 668f5a620b0c43da9d16d9ecb463caa8 P 4ae03fe546924cad854258eb297dc52a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:03:26.180190 17903 log.cc:1196] T 668f5a620b0c43da9d16d9ecb463caa8 P 4ae03fe546924cad854258eb297dc52a: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/668f5a620b0c43da9d16d9ecb463caa8
I20260926 15:03:26.180583 17903 ts_tablet_manager.cc:1939] T 668f5a620b0c43da9d16d9ecb463caa8 P 4ae03fe546924cad854258eb297dc52a: Deleting consensus metadata
I20260926 15:03:26.180567 17902 ts_tablet_manager.cc:1918] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:03:26.180727 17902 log.cc:1196] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-0/wal/wals/668f5a620b0c43da9d16d9ecb463caa8
I20260926 15:03:26.181077 17902 ts_tablet_manager.cc:1939] T 668f5a620b0c43da9d16d9ecb463caa8 P e5a1a54cb63c445380be679d749f8c62: Deleting consensus metadata
I20260926 15:03:26.182020 17305 catalog_manager.cc:4841] TS 4ae03fe546924cad854258eb297dc52a (127.9.123.66:34747): tablet 668f5a620b0c43da9d16d9ecb463caa8 (table test-table [id=49f77fabef0c4aa49d7ecf4493c2a927]) successfully deleted
I20260926 15:03:26.182605 17304 catalog_manager.cc:4841] TS e5a1a54cb63c445380be679d749f8c62 (127.9.123.65:33917): tablet 668f5a620b0c43da9d16d9ecb463caa8 (table test-table [id=49f77fabef0c4aa49d7ecf4493c2a927]) successfully deleted
I20260926 15:03:26.185132 17457 tablet_service.cc:1467] Processing CreateTablet for tablet 6722e0e0f3404a239067ded4d1c810d7 (DEFAULT_TABLE table=test-table [id=be897355dfc543159849b00bb97f4775]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:03:26.185333 17596 tablet_service.cc:1467] Processing CreateTablet for tablet 6722e0e0f3404a239067ded4d1c810d7 (DEFAULT_TABLE table=test-table [id=be897355dfc543159849b00bb97f4775]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:03:26.185545 17457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6722e0e0f3404a239067ded4d1c810d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:26.185542 17904 ts_tablet_manager.cc:1918] T 668f5a620b0c43da9d16d9ecb463caa8 P e9d05cc314cb404086e36463718888a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:03:26.185695 17904 log.cc:1196] T 668f5a620b0c43da9d16d9ecb463caa8 P e9d05cc314cb404086e36463718888a5: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/668f5a620b0c43da9d16d9ecb463caa8
I20260926 15:03:26.185751 17596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6722e0e0f3404a239067ded4d1c810d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:26.186110 17904 ts_tablet_manager.cc:1939] T 668f5a620b0c43da9d16d9ecb463caa8 P e9d05cc314cb404086e36463718888a5: Deleting consensus metadata
I20260926 15:03:26.189931 17832 tablet_bootstrap.cc:492] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62: Bootstrap starting.
I20260926 15:03:26.189981 17304 catalog_manager.cc:4841] TS e9d05cc314cb404086e36463718888a5 (127.9.123.67:42353): tablet 668f5a620b0c43da9d16d9ecb463caa8 (table test-table [id=49f77fabef0c4aa49d7ecf4493c2a927]) successfully deleted
I20260926 15:03:26.190315 17735 tablet_service.cc:1467] Processing CreateTablet for tablet 6722e0e0f3404a239067ded4d1c810d7 (DEFAULT_TABLE table=test-table [id=be897355dfc543159849b00bb97f4775]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:03:26.190744 17735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6722e0e0f3404a239067ded4d1c810d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:26.191094 17830 tablet_bootstrap.cc:492] T 6722e0e0f3404a239067ded4d1c810d7 P 4ae03fe546924cad854258eb297dc52a: Bootstrap starting.
I20260926 15:03:26.191426 17832 tablet_bootstrap.cc:654] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:26.192625 17830 tablet_bootstrap.cc:654] T 6722e0e0f3404a239067ded4d1c810d7 P 4ae03fe546924cad854258eb297dc52a: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:26.194134 17832 tablet_bootstrap.cc:492] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62: No bootstrap required, opened a new log
I20260926 15:03:26.194295 17832 ts_tablet_manager.cc:1397] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260926 15:03:26.194996 17830 tablet_bootstrap.cc:492] T 6722e0e0f3404a239067ded4d1c810d7 P 4ae03fe546924cad854258eb297dc52a: No bootstrap required, opened a new log
I20260926 15:03:26.195143 17830 ts_tablet_manager.cc:1397] T 6722e0e0f3404a239067ded4d1c810d7 P 4ae03fe546924cad854258eb297dc52a: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260926 15:03:26.195199 17832 raft_consensus.cc:348] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:26.195431 17832 raft_consensus.cc:374] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:26.195497 17832 raft_consensus.cc:729] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5a1a54cb63c445380be679d749f8c62, State: Initialized, Role: FOLLOWER
I20260926 15:03:26.195730 17832 consensus_queue.cc:260] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:26.195868 17830 raft_consensus.cc:348] T 6722e0e0f3404a239067ded4d1c810d7 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:26.196087 17830 raft_consensus.cc:374] T 6722e0e0f3404a239067ded4d1c810d7 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:26.196156 17830 raft_consensus.cc:729] T 6722e0e0f3404a239067ded4d1c810d7 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ae03fe546924cad854258eb297dc52a, State: Initialized, Role: FOLLOWER
I20260926 15:03:26.196381 17830 consensus_queue.cc:260] T 6722e0e0f3404a239067ded4d1c810d7 P 4ae03fe546924cad854258eb297dc52a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:26.196257 17832 ts_tablet_manager.cc:1428] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260926 15:03:26.196893 17830 ts_tablet_manager.cc:1428] T 6722e0e0f3404a239067ded4d1c810d7 P 4ae03fe546924cad854258eb297dc52a: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260926 15:03:26.197479 17831 tablet_bootstrap.cc:492] T 6722e0e0f3404a239067ded4d1c810d7 P e9d05cc314cb404086e36463718888a5: Bootstrap starting.
I20260926 15:03:26.199252 17831 tablet_bootstrap.cc:654] T 6722e0e0f3404a239067ded4d1c810d7 P e9d05cc314cb404086e36463718888a5: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:26.203341 17831 tablet_bootstrap.cc:492] T 6722e0e0f3404a239067ded4d1c810d7 P e9d05cc314cb404086e36463718888a5: No bootstrap required, opened a new log
I20260926 15:03:26.203553 17831 ts_tablet_manager.cc:1397] T 6722e0e0f3404a239067ded4d1c810d7 P e9d05cc314cb404086e36463718888a5: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260926 15:03:26.204339 17831 raft_consensus.cc:348] T 6722e0e0f3404a239067ded4d1c810d7 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:26.204631 17831 raft_consensus.cc:374] T 6722e0e0f3404a239067ded4d1c810d7 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:26.204711 17831 raft_consensus.cc:729] T 6722e0e0f3404a239067ded4d1c810d7 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d05cc314cb404086e36463718888a5, State: Initialized, Role: FOLLOWER
I20260926 15:03:26.204931 17831 consensus_queue.cc:260] T 6722e0e0f3404a239067ded4d1c810d7 P e9d05cc314cb404086e36463718888a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:26.205387 17831 ts_tablet_manager.cc:1428] T 6722e0e0f3404a239067ded4d1c810d7 P e9d05cc314cb404086e36463718888a5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260926 15:03:26.342341 17901 raft_consensus.cc:482] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:03:26.342602 17901 raft_consensus.cc:504] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:26.343180 17901 leader_election.cc:288] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ae03fe546924cad854258eb297dc52a (127.9.123.66:34747), e9d05cc314cb404086e36463718888a5 (127.9.123.67:42353)
I20260926 15:03:26.343746 17616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6722e0e0f3404a239067ded4d1c810d7" candidate_uuid: "e5a1a54cb63c445380be679d749f8c62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ae03fe546924cad854258eb297dc52a" is_pre_election: true
I20260926 15:03:26.343761 17755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6722e0e0f3404a239067ded4d1c810d7" candidate_uuid: "e5a1a54cb63c445380be679d749f8c62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d05cc314cb404086e36463718888a5" is_pre_election: true
I20260926 15:03:26.343963 17616 raft_consensus.cc:2454] T 6722e0e0f3404a239067ded4d1c810d7 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5a1a54cb63c445380be679d749f8c62 in term 0.
I20260926 15:03:26.343984 17755 raft_consensus.cc:2454] T 6722e0e0f3404a239067ded4d1c810d7 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5a1a54cb63c445380be679d749f8c62 in term 0.
I20260926 15:03:26.344352 17408 leader_election.cc:302] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ae03fe546924cad854258eb297dc52a, e5a1a54cb63c445380be679d749f8c62; no voters:
I20260926 15:03:26.344655 17901 raft_consensus.cc:2781] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:03:26.344734 17901 raft_consensus.cc:482] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:03:26.344822 17901 raft_consensus.cc:3037] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:26.346774 17901 raft_consensus.cc:504] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:26.347352 17901 leader_election.cc:288] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62 [CANDIDATE]: Term 1 election: Requested vote from peers 4ae03fe546924cad854258eb297dc52a (127.9.123.66:34747), e9d05cc314cb404086e36463718888a5 (127.9.123.67:42353)
I20260926 15:03:26.347822 17616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6722e0e0f3404a239067ded4d1c810d7" candidate_uuid: "e5a1a54cb63c445380be679d749f8c62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ae03fe546924cad854258eb297dc52a"
I20260926 15:03:26.347899 17755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6722e0e0f3404a239067ded4d1c810d7" candidate_uuid: "e5a1a54cb63c445380be679d749f8c62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d05cc314cb404086e36463718888a5"
I20260926 15:03:26.348016 17616 raft_consensus.cc:3037] T 6722e0e0f3404a239067ded4d1c810d7 P 4ae03fe546924cad854258eb297dc52a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:26.348106 17755 raft_consensus.cc:3037] T 6722e0e0f3404a239067ded4d1c810d7 P e9d05cc314cb404086e36463718888a5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:26.349969 17616 raft_consensus.cc:2454] T 6722e0e0f3404a239067ded4d1c810d7 P 4ae03fe546924cad854258eb297dc52a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5a1a54cb63c445380be679d749f8c62 in term 1.
I20260926 15:03:26.350147 17755 raft_consensus.cc:2454] T 6722e0e0f3404a239067ded4d1c810d7 P e9d05cc314cb404086e36463718888a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5a1a54cb63c445380be679d749f8c62 in term 1.
I20260926 15:03:26.350395 17408 leader_election.cc:302] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ae03fe546924cad854258eb297dc52a, e5a1a54cb63c445380be679d749f8c62; no voters:
I20260926 15:03:26.350647 17901 raft_consensus.cc:2781] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:03:26.350805 17901 raft_consensus.cc:686] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62 [term 1 LEADER]: Becoming Leader. State: Replica: e5a1a54cb63c445380be679d749f8c62, State: Running, Role: LEADER
I20260926 15:03:26.351058 17901 consensus_queue.cc:237] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } }
I20260926 15:03:26.353871 17318 catalog_manager.cc:5495] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62 reported cstate change: term changed from 0 to 1, leader changed from <none> to e5a1a54cb63c445380be679d749f8c62 (127.9.123.65). New cstate: current_term: 1 leader_uuid: "e5a1a54cb63c445380be679d749f8c62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5a1a54cb63c445380be679d749f8c62" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 } health_report { overall_health: UNKNOWN } } }
I20260926 15:03:26.409838 17616 raft_consensus.cc:1261] T 6722e0e0f3404a239067ded4d1c810d7 P 4ae03fe546924cad854258eb297dc52a [term 1 FOLLOWER]: Refusing update from remote peer e5a1a54cb63c445380be679d749f8c62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:03:26.409952 17755 raft_consensus.cc:1261] T 6722e0e0f3404a239067ded4d1c810d7 P e9d05cc314cb404086e36463718888a5 [term 1 FOLLOWER]: Refusing update from remote peer e5a1a54cb63c445380be679d749f8c62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:03:26.410355 17837 consensus_queue.cc:1035] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ae03fe546924cad854258eb297dc52a" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:03:26.410717 17901 consensus_queue.cc:1035] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9d05cc314cb404086e36463718888a5" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 42353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:03:26.457747 17318 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:03:26.457973 17318 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:03:26.460714 17318 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6315f47bfa9454faf8479114234aa07: Sending DeleteTablet for 3 replicas of tablet 6722e0e0f3404a239067ded4d1c810d7
I20260926 15:03:26.461506 17596 tablet_service.cc:1514] Processing DeleteTablet for tablet 6722e0e0f3404a239067ded4d1c810d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:03:26 UTC) from {username='oryx', principal='oryx/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:55700
I20260926 15:03:26.461526 17735 tablet_service.cc:1514] Processing DeleteTablet for tablet 6722e0e0f3404a239067ded4d1c810d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:03:26 UTC) from {username='oryx', principal='oryx/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:60510
I20260926 15:03:26.461519 17457 tablet_service.cc:1514] Processing DeleteTablet for tablet 6722e0e0f3404a239067ded4d1c810d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:03:26 UTC) from {username='oryx', principal='oryx/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:37844
I20260926 15:03:26.461794 17904 tablet_replica.cc:330] T 6722e0e0f3404a239067ded4d1c810d7 P e9d05cc314cb404086e36463718888a5: stopping tablet replica
I20260926 15:03:26.461794 17902 tablet_replica.cc:330] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62: stopping tablet replica
I20260926 15:03:26.461818 17903 tablet_replica.cc:330] T 6722e0e0f3404a239067ded4d1c810d7 P 4ae03fe546924cad854258eb297dc52a: stopping tablet replica
I20260926 15:03:26.461949 17902 raft_consensus.cc:2229] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62 [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:03:26.461961 17904 raft_consensus.cc:2229] T 6722e0e0f3404a239067ded4d1c810d7 P e9d05cc314cb404086e36463718888a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:03:26.461989 17903 raft_consensus.cc:2229] T 6722e0e0f3404a239067ded4d1c810d7 P 4ae03fe546924cad854258eb297dc52a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:03:26.462139 17904 raft_consensus.cc:2258] T 6722e0e0f3404a239067ded4d1c810d7 P e9d05cc314cb404086e36463718888a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:26.462175 17903 raft_consensus.cc:2258] T 6722e0e0f3404a239067ded4d1c810d7 P 4ae03fe546924cad854258eb297dc52a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:26.462247 17902 raft_consensus.cc:2258] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:26.463248 17904 ts_tablet_manager.cc:1905] T 6722e0e0f3404a239067ded4d1c810d7 P e9d05cc314cb404086e36463718888a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:03:26.463349 17903 ts_tablet_manager.cc:1905] T 6722e0e0f3404a239067ded4d1c810d7 P 4ae03fe546924cad854258eb297dc52a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:03:26.463610 17902 ts_tablet_manager.cc:1905] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:03:26.468711 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 17391
I20260926 15:03:26.469200 17902 ts_tablet_manager.cc:1918] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 15:03:26.469359 17902 log.cc:1196] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-0/wal/wals/6722e0e0f3404a239067ded4d1c810d7
I20260926 15:03:26.469777 17902 ts_tablet_manager.cc:1939] T 6722e0e0f3404a239067ded4d1c810d7 P e5a1a54cb63c445380be679d749f8c62: Deleting consensus metadata
I20260926 15:03:26.469879 17904 ts_tablet_manager.cc:1918] T 6722e0e0f3404a239067ded4d1c810d7 P e9d05cc314cb404086e36463718888a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 15:03:26.469879 17903 ts_tablet_manager.cc:1918] T 6722e0e0f3404a239067ded4d1c810d7 P 4ae03fe546924cad854258eb297dc52a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260926 15:03:26.470036 17903 log.cc:1196] T 6722e0e0f3404a239067ded4d1c810d7 P 4ae03fe546924cad854258eb297dc52a: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/6722e0e0f3404a239067ded4d1c810d7
I20260926 15:03:26.470147 17904 log.cc:1196] T 6722e0e0f3404a239067ded4d1c810d7 P e9d05cc314cb404086e36463718888a5: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/6722e0e0f3404a239067ded4d1c810d7
I20260926 15:03:26.470441 17903 ts_tablet_manager.cc:1939] T 6722e0e0f3404a239067ded4d1c810d7 P 4ae03fe546924cad854258eb297dc52a: Deleting consensus metadata
I20260926 15:03:26.470505 17904 ts_tablet_manager.cc:1939] T 6722e0e0f3404a239067ded4d1c810d7 P e9d05cc314cb404086e36463718888a5: Deleting consensus metadata
I20260926 15:03:26.471495 17305 catalog_manager.cc:4841] TS 4ae03fe546924cad854258eb297dc52a (127.9.123.66:34747): tablet 6722e0e0f3404a239067ded4d1c810d7 (table test-table [id=be897355dfc543159849b00bb97f4775]) successfully deleted
I20260926 15:03:26.471585 17304 catalog_manager.cc:4841] TS e9d05cc314cb404086e36463718888a5 (127.9.123.67:42353): tablet 6722e0e0f3404a239067ded4d1c810d7 (table test-table [id=be897355dfc543159849b00bb97f4775]) successfully deleted
I20260926 15:03:26.474152 17304 catalog_manager.cc:4841] TS e5a1a54cb63c445380be679d749f8c62 (127.9.123.65:33917): tablet 6722e0e0f3404a239067ded4d1c810d7 (table test-table [id=be897355dfc543159849b00bb97f4775]) successfully deleted
Sep 26 15:03:26 dist-test-slave-7153 krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.123.65@KRBTEST.COM
I20260926 15:03:26.488379 17874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:26.470157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37874 (local address 127.9.123.65:33917)
0926 15:03:26.470327 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:26.470333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:26.474404 (+ 4071us) server_negotiation.cc:407] Connection header received
0926 15:03:26.474614 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:26.474618 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:26.474695 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:26.474789 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:26.475943 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:26.476688 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:26.477861 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:26.478265 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:26.478366 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:26.482165 (+ 3799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:26.482180 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:26.482196 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:26.482240 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:26.485463 (+ 3223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:26.486269 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:26.486278 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:26.486280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:26.486437 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:26.486827 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:26.486832 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:26.486834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:26.487294 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:26.487556 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:26.487882 (+ 326us) server_negotiation.cc:299] Negotiation successful
0926 15:03:26.488064 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 15:03:26.489082 17522 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:26.611440 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 17391
I20260926 15:03:26.630468 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 17531
Sep 26 15:03:26 dist-test-slave-7153 krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.123.66@KRBTEST.COM
I20260926 15:03:26.644618 17936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:26.631165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55720 (local address 127.9.123.66:34747)
0926 15:03:26.631638 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:26.631642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:26.631657 (+ 15us) server_negotiation.cc:407] Connection header received
0926 15:03:26.631708 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:26.631712 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:26.631773 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:26.631859 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:26.632936 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:26.633654 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:26.634934 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:26.635331 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:26.635419 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:26.639575 (+ 4156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:26.639590 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:26.639606 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:26.639639 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:26.642420 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:26.643039 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:26.643046 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:26.643049 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:26.643164 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:26.643432 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:26.643435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:26.643437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:26.643753 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:26.643959 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:26.644150 (+ 191us) server_negotiation.cc:299] Negotiation successful
0926 15:03:26.644302 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":232,"threads_started":1}
I20260926 15:03:26.645254 17661 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:26.763993 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 17531
I20260926 15:03:26.781466 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 17670
Sep 26 15:03:26 dist-test-slave-7153 krb5kdc[17275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.123.67@KRBTEST.COM
I20260926 15:03:26.796667 17938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:26.782223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60518 (local address 127.9.123.67:42353)
0926 15:03:26.782716 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:26.782721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:26.782738 (+ 17us) server_negotiation.cc:407] Connection header received
0926 15:03:26.782793 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:26.782796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:26.782860 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:26.782966 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:26.784201 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:26.784985 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:26.786222 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:26.786638 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:26.786733 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:26.790888 (+ 4155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:26.790905 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:26.790922 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:26.790967 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:26.793725 (+ 2758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:26.794432 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:26.794441 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:26.794444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:26.794635 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:26.795034 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:26.795039 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:26.795041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:26.795508 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:26.795773 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:26.796102 (+ 329us) server_negotiation.cc:299] Negotiation successful
0926 15:03:26.796288 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":286,"threads_started":1}
I20260926 15:03:26.797425 17799 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:26.894048 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 17670
I20260926 15:03:26.908313 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 17288
I20260926 15:03:26.909478 17360 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:27.022756 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 17288
2026-09-26T15:03:27Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6519 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:03:27 dist-test-slave-7153 krb5kdc[17943](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:03:27 dist-test-slave-7153 krb5kdc[17943](info): set up 2 sockets
Sep 26 15:03:27 dist-test-slave-7153 krb5kdc[17943](info): commencing operation
krb5kdc: starting...
W20260926 15:03:28.041945 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.979s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:03:28 dist-test-slave-7153 krb5kdc[17943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:03:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:03:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:03:28.261247 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:35921
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:41861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:35921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:03:28.597759 17956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:28.598064 17956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:28.598181 17956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:28.605577 17956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:03:28.605701 17956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:28.605757 17956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:28.605835 17956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:03:28.605896 17956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:03:28.616084 17956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:41861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:35921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:35921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:28.617489 17956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:28.618870 17956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:28.630506 17961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:28.630577 17962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:28.630543 17964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:28.631697 17956 server_base.cc:1034] running on GCE node
I20260926 15:03:28.632507 17956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:28.634051 17956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:28.635303 17956 hybrid_clock.cc:648] HybridClock initialized: now 1790435008635252 us; error 40 us; skew 500 ppm
I20260926 15:03:28.635705 17956 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-26T15:03:28Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1679 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:03:28 dist-test-slave-7153 krb5kdc[17970](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:03:28 dist-test-slave-7153 krb5kdc[17970](info): set up 2 sockets
Sep 26 15:03:28 dist-test-slave-7153 krb5kdc[17970](info): commencing operation
krb5kdc: starting...
W20260926 15:03:29.725795 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.983s user 0.001s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:03:29 dist-test-slave-7153 krb5kdc[17970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:03:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:03:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:03:29.940403 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:39341
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:46001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:39341
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:03:30.261713 17983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:30.262023 17983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:30.262151 17983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:30.269558 17983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:03:30.269673 17983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:30.269738 17983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:30.269814 17983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:03:30.269871 17983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:03:30.279783 17983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:46001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:39341
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:39341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:30.281157 17983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:30.282472 17983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:30.293829 17991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:30.293804 17988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:30.293890 17989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:30.294607 17983 server_base.cc:1034] running on GCE node
I20260926 15:03:30.295429 17983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:30.296891 17983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:30.298136 17983 hybrid_clock.cc:648] HybridClock initialized: now 1790435010298057 us; error 65 us; skew 500 ppm
I20260926 15:03:30.298509 17983 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:30 dist-test-slave-7153 krb5kdc[17970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435010, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:30.306530 17983 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:03:30.308111 17983 webserver.cc:460] Webserver started at http://127.9.123.126:34469/ using document root <none> and password file <none>
I20260926 15:03:30.308837 17983 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:30.308945 17983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:30.309275 17983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:30.312305 17983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "3330df1fe5094f1b8130bf29b070817b"
format_stamp: "Formatted at 2026-09-26 15:03:30 on dist-test-slave-7153"
server_key: "b62e88451852198648991f5c0fe449fe"
server_key_iv: "76cfb1808e33f330ee40300bd89f943e"
server_key_version: "encryptionkey@0"
I20260926 15:03:30.313028 17983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "3330df1fe5094f1b8130bf29b070817b"
format_stamp: "Formatted at 2026-09-26 15:03:30 on dist-test-slave-7153"
server_key: "b62e88451852198648991f5c0fe449fe"
server_key_iv: "76cfb1808e33f330ee40300bd89f943e"
server_key_version: "encryptionkey@0"
I20260926 15:03:30.318114 17983 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 15:03:30.321542 17998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:30.322984 17983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260926 15:03:30.323148 17983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "3330df1fe5094f1b8130bf29b070817b"
format_stamp: "Formatted at 2026-09-26 15:03:30 on dist-test-slave-7153"
server_key: "b62e88451852198648991f5c0fe449fe"
server_key_iv: "76cfb1808e33f330ee40300bd89f943e"
server_key_version: "encryptionkey@0"
I20260926 15:03:30.323325 17983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:30.341609 17983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:30.345961 17983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:30.346220 17983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:30.362884 17983 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:39341
I20260926 15:03:30.362902 18049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:39341 every 8 connection(s)
I20260926 15:03:30.364266 17983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:03:30.368054 18050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:30.371310 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 17983
I20260926 15:03:30.371541 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:03:30.372229 9709 external_mini_cluster.cc:1428] Setting key 9c04a26f327833ac62b3357625ce63d4
I20260926 15:03:30.380306 18050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b: Bootstrap starting.
I20260926 15:03:30.383534 18050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:30.384560 18050 log.cc:824] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b: Log is configured to *not* fsync() on all Append() calls
Sep 26 15:03:30 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:03:30.387878 18050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b: No bootstrap required, opened a new log
I20260926 15:03:30.394506 18053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:30.374189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58382 (local address 127.9.123.126:39341)
0926 15:03:30.375002 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:30.375015 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:30.375060 (+ 45us) server_negotiation.cc:407] Connection header received
0926 15:03:30.376756 (+ 1696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:30.376783 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:30.377340 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:30.377757 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:30.379231 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:30.380619 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:30.381989 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:30.382688 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:30.382881 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:30.386840 (+ 3959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:30.386848 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:30.386864 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:30.386905 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:30.390094 (+ 3189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:30.391039 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:30.391046 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:30.391057 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:30.391231 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:30.391625 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:30.391628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:30.391630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:30.392209 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:30.392458 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:30.392900 (+ 442us) server_negotiation.cc:299] Negotiation successful
0926 15:03:30.393353 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":360,"threads_started":1}
I20260926 15:03:30.396876 18050 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3330df1fe5094f1b8130bf29b070817b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39341 } }
I20260926 15:03:30.397413 18050 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:30.397527 18050 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3330df1fe5094f1b8130bf29b070817b, State: Initialized, Role: FOLLOWER
I20260926 15:03:30.398207 18050 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3330df1fe5094f1b8130bf29b070817b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39341 } }
I20260926 15:03:30.398442 18050 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:03:30.398540 18050 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:03:30.398656 18050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:30.400786 18050 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3330df1fe5094f1b8130bf29b070817b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39341 } }
I20260926 15:03:30.401337 18050 leader_election.cc:302] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3330df1fe5094f1b8130bf29b070817b; no voters:
I20260926 15:03:30.401921 18050 leader_election.cc:288] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:03:30.402069 18055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:03:30.402695 18055 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [term 1 LEADER]: Becoming Leader. State: Replica: 3330df1fe5094f1b8130bf29b070817b, State: Running, Role: LEADER
I20260926 15:03:30.403386 18055 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3330df1fe5094f1b8130bf29b070817b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39341 } }
I20260926 15:03:30.404623 18050 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:03:30.406232 18056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3330df1fe5094f1b8130bf29b070817b. Latest consensus state: current_term: 1 leader_uuid: "3330df1fe5094f1b8130bf29b070817b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3330df1fe5094f1b8130bf29b070817b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39341 } } }
I20260926 15:03:30.406276 18057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3330df1fe5094f1b8130bf29b070817b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3330df1fe5094f1b8130bf29b070817b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39341 } } }
I20260926 15:03:30.406633 18056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:30.406634 18057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:30.408975 18061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:03:30.414585 18061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:03:30.423547 18061 catalog_manager.cc:1347] Generated new cluster ID: 8bcbf891b77648ddb1fbce9e21de5f43
I20260926 15:03:30.423727 18061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:03:30.433643 18061 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:03:30.435258 18061 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:03:30.443352 18061 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b: Generated new TSK 0
I20260926 15:03:30.444217 18061 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.123.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
I20260926 15:03:30.501065 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:39341
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:46001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:03:30.826138 18076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:30.826498 18076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:30.826642 18076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:30.834244 18076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:30.834368 18076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:30.834515 18076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:03:30.844313 18076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:46001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:39341
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:30.845705 18076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:30.847059 18076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:30.860584 18081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:30.860670 18084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:30.860683 18082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:30.861562 18076 server_base.cc:1034] running on GCE node
I20260926 15:03:30.862182 18076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:30.863147 18076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:30.864358 18076 hybrid_clock.cc:648] HybridClock initialized: now 1790435010864301 us; error 46 us; skew 500 ppm
I20260926 15:03:30.864667 18076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:30 dist-test-slave-7153 krb5kdc[17970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435010, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:30.867892 18076 init.cc:377] Logged in from keytab as kudu/127.9.123.65@KRBTEST.COM (short username kudu)
I20260926 15:03:30.869482 18076 webserver.cc:460] Webserver started at http://127.9.123.65:44715/ using document root <none> and password file <none>
I20260926 15:03:30.870184 18076 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:30.870293 18076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:30.870586 18076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:30.873580 18076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "c66fea23b1e24343a9a0a6b4f384c7b8"
format_stamp: "Formatted at 2026-09-26 15:03:30 on dist-test-slave-7153"
server_key: "31670a39ade887b76c9eb02f51f7e4ce"
server_key_iv: "2efadcff7147b6b6320d4e1e3db874b3"
server_key_version: "encryptionkey@0"
I20260926 15:03:30.874368 18076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "c66fea23b1e24343a9a0a6b4f384c7b8"
format_stamp: "Formatted at 2026-09-26 15:03:30 on dist-test-slave-7153"
server_key: "31670a39ade887b76c9eb02f51f7e4ce"
server_key_iv: "2efadcff7147b6b6320d4e1e3db874b3"
server_key_version: "encryptionkey@0"
I20260926 15:03:30.879789 18076 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 15:03:30.883741 18091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:30.885345 18076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 15:03:30.885509 18076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "c66fea23b1e24343a9a0a6b4f384c7b8"
format_stamp: "Formatted at 2026-09-26 15:03:30 on dist-test-slave-7153"
server_key: "31670a39ade887b76c9eb02f51f7e4ce"
server_key_iv: "2efadcff7147b6b6320d4e1e3db874b3"
server_key_version: "encryptionkey@0"
I20260926 15:03:30.885704 18076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:30.911969 18076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:30.916453 18076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:30.916742 18076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:30.917745 18076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:30.919104 18076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:30.919185 18076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:30.919301 18076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:30.919350 18076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:30.949910 18076 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:41845
I20260926 15:03:30.949960 18203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:41845 every 8 connection(s)
I20260926 15:03:30.951156 18076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:03:30.952454 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 18076
I20260926 15:03:30.952636 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:03:30.953303 9709 external_mini_cluster.cc:1428] Setting key 1b4d201387c2ad9d46b49a057bddcee4
Sep 26 15:03:30 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435010, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:03:30.971729 18206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:30.953732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:47483 (local address 127.9.123.126:39341)
0926 15:03:30.954258 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:30.954264 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:30.955284 (+ 1020us) server_negotiation.cc:407] Connection header received
0926 15:03:30.957136 (+ 1852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:30.957141 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:30.957232 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:30.957322 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:30.958945 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:30.959703 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:30.961715 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:30.962127 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:30.962238 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:30.966136 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:30.966141 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:30.966145 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:30.966178 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:30.968523 (+ 2345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:30.969324 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:30.969330 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:30.969334 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:30.969481 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:30.969885 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:30.969891 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:30.969894 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:30.970255 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:30.970407 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:30.971124 (+ 717us) server_negotiation.cc:299] Negotiation successful
0926 15:03:30.971353 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":271,"threads_started":1}
I20260926 15:03:30.972502 18207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:30.954341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.65:47483)
0926 15:03:30.955134 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:30.955190 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:30.956844 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:30.957590 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:30.957599 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:30.958136 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:30.958755 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:30.958775 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:30.959872 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:30.959878 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:30.961517 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:30.961558 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:30.962366 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:30.962372 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:30.962667 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:30.963375 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:30.963403 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:30.965892 (+ 2489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:30.968672 (+ 2780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:30.968682 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:30.968701 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:30.969185 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:30.969586 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:30.969589 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:30.969591 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:30.969766 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:30.970371 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:30.970377 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:30.970739 (+ 362us) client_negotiation.cc:769] Sending connection context
0926 15:03:30.970967 (+ 228us) client_negotiation.cc:240] Negotiation successful
0926 15:03:30.971365 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":298,"threads_started":1}
WARNING: no policy specified for kudu/127.9.123.66@KRBTEST.COM; defaulting to no policy
I20260926 15:03:30.974059 18204 heartbeater.cc:347] Connected to a master server at 127.9.123.126:39341
I20260926 15:03:30.974326 18204 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:30.975090 18204 heartbeater.cc:511] Master 127.9.123.126:39341 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.66@KRBTEST.COM" created.
I20260926 15:03:30.977303 18015 ts_manager.cc:194] Registered new tserver with Master: c66fea23b1e24343a9a0a6b4f384c7b8 (127.9.123.65:41845)
I20260926 15:03:30.978744 18015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.65@KRBTEST.COM'} at 127.9.123.65:47483
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
I20260926 15:03:31.001403 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:39341
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:46001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:03:31.326319 18210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:31.326640 18210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:31.326781 18210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:31.334589 18210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:31.334719 18210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:31.334870 18210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:03:31.344818 18210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:46001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:39341
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:31.346161 18210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:31.347527 18210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:31.360535 18216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:31.360473 18215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:31.360473 18218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:31.361267 18210 server_base.cc:1034] running on GCE node
I20260926 15:03:31.361860 18210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:31.362789 18210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:31.364007 18210 hybrid_clock.cc:648] HybridClock initialized: now 1790435011363936 us; error 66 us; skew 500 ppm
I20260926 15:03:31.364313 18210 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:31 dist-test-slave-7153 krb5kdc[17970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435011, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:31.367446 18210 init.cc:377] Logged in from keytab as kudu/127.9.123.66@KRBTEST.COM (short username kudu)
I20260926 15:03:31.369026 18210 webserver.cc:460] Webserver started at http://127.9.123.66:40259/ using document root <none> and password file <none>
I20260926 15:03:31.369778 18210 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:31.369895 18210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:31.370196 18210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:31.373178 18210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "9bed0958116747ada838459e0316cf85"
format_stamp: "Formatted at 2026-09-26 15:03:31 on dist-test-slave-7153"
server_key: "5b541453bb6587492486761126ce65b7"
server_key_iv: "e379968bb30a4c18483aa088ddfc4eda"
server_key_version: "encryptionkey@0"
I20260926 15:03:31.374042 18210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "9bed0958116747ada838459e0316cf85"
format_stamp: "Formatted at 2026-09-26 15:03:31 on dist-test-slave-7153"
server_key: "5b541453bb6587492486761126ce65b7"
server_key_iv: "e379968bb30a4c18483aa088ddfc4eda"
server_key_version: "encryptionkey@0"
I20260926 15:03:31.379477 18210 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 15:03:31.383291 18225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:31.384706 18210 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260926 15:03:31.384860 18210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "9bed0958116747ada838459e0316cf85"
format_stamp: "Formatted at 2026-09-26 15:03:31 on dist-test-slave-7153"
server_key: "5b541453bb6587492486761126ce65b7"
server_key_iv: "e379968bb30a4c18483aa088ddfc4eda"
server_key_version: "encryptionkey@0"
I20260926 15:03:31.385051 18210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:31.403118 18210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:31.407668 18210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:31.407943 18210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:31.408915 18210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:31.410374 18210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:31.410447 18210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:31.410531 18210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:31.410580 18210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:31.440773 18210 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:42873
I20260926 15:03:31.440815 18337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:42873 every 8 connection(s)
I20260926 15:03:31.442037 18210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:03:31.452435 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 18210
I20260926 15:03:31.452667 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:03:31.453233 9709 external_mini_cluster.cc:1428] Setting key 717e3e79914fad630eac5c3b0ce44f9d
Sep 26 15:03:31 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435011, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:03:31.462989 18206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:31.444494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:52371 (local address 127.9.123.126:39341)
0926 15:03:31.444658 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:31.444663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:31.445825 (+ 1162us) server_negotiation.cc:407] Connection header received
0926 15:03:31.447602 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:31.447607 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:31.447675 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:31.447758 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:31.449359 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:31.450078 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:31.451819 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:31.452286 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:31.452460 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:31.456915 (+ 4455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:31.456921 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:31.456925 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:31.456964 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:31.459546 (+ 2582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:31.460484 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:31.460488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:31.460490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:31.460618 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:31.461079 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:31.461081 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:31.461083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:31.461406 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:31.461519 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:31.462374 (+ 855us) server_negotiation.cc:299] Negotiation successful
0926 15:03:31.462650 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 15:03:31.463786 18340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:31.444938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.66:52371)
0926 15:03:31.445665 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:31.445704 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:31.447340 (+ 1636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:31.447964 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:31.447972 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:31.448537 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:31.449184 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:31.449210 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:31.450249 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:31.450253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:31.451625 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:31.451639 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:31.452667 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:31.452676 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:31.453026 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:31.453845 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:31.453876 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:31.456634 (+ 2758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:31.459714 (+ 3080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:31.459725 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:31.459746 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:31.460351 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:31.460724 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:31.460729 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:31.460732 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:31.460983 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:31.461518 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:31.461527 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:31.461993 (+ 466us) client_negotiation.cc:769] Sending connection context
0926 15:03:31.462228 (+ 235us) client_negotiation.cc:240] Negotiation successful
0926 15:03:31.462644 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":292,"threads_started":1}
I20260926 15:03:31.465296 18338 heartbeater.cc:347] Connected to a master server at 127.9.123.126:39341
I20260926 15:03:31.465634 18338 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:31.466516 18338 heartbeater.cc:511] Master 127.9.123.126:39341 requested a full tablet report, sending...
I20260926 15:03:31.468140 18015 ts_manager.cc:194] Registered new tserver with Master: 9bed0958116747ada838459e0316cf85 (127.9.123.66:42873)
I20260926 15:03:31.468952 18015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.66@KRBTEST.COM'} at 127.9.123.66:52371
WARNING: no policy specified for kudu/127.9.123.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
I20260926 15:03:31.505909 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:39341
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:46001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:03:31.837199 18343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:31.837507 18343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:31.837628 18343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:31.845037 18343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:31.845208 18343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:31.845363 18343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:03:31.855397 18343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:46001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:39341
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:31.856824 18343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:31.858157 18343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:31.870999 18348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:31.870961 18351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:31.870929 18349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:31.871876 18343 server_base.cc:1034] running on GCE node
I20260926 15:03:31.872483 18343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:31.873500 18343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:31.874708 18343 hybrid_clock.cc:648] HybridClock initialized: now 1790435011874660 us; error 49 us; skew 500 ppm
I20260926 15:03:31.875041 18343 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:31 dist-test-slave-7153 krb5kdc[17970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435011, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:31.878170 18343 init.cc:377] Logged in from keytab as kudu/127.9.123.67@KRBTEST.COM (short username kudu)
I20260926 15:03:31.879683 18343 webserver.cc:460] Webserver started at http://127.9.123.67:40139/ using document root <none> and password file <none>
I20260926 15:03:31.880393 18343 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:31.880513 18343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:31.880820 18343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:31.883886 18343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "dd481ee0cb2b4e089e26e74af8ecf5b4"
format_stamp: "Formatted at 2026-09-26 15:03:31 on dist-test-slave-7153"
server_key: "01ef7e956bbb84e4767b6d6ccf5b7804"
server_key_iv: "8d05d6fc2ffd5dfb1db563a6076aa069"
server_key_version: "encryptionkey@0"
I20260926 15:03:31.884702 18343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "dd481ee0cb2b4e089e26e74af8ecf5b4"
format_stamp: "Formatted at 2026-09-26 15:03:31 on dist-test-slave-7153"
server_key: "01ef7e956bbb84e4767b6d6ccf5b7804"
server_key_iv: "8d05d6fc2ffd5dfb1db563a6076aa069"
server_key_version: "encryptionkey@0"
I20260926 15:03:31.890765 18343 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260926 15:03:31.895016 18358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:31.896652 18343 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 15:03:31.896855 18343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "dd481ee0cb2b4e089e26e74af8ecf5b4"
format_stamp: "Formatted at 2026-09-26 15:03:31 on dist-test-slave-7153"
server_key: "01ef7e956bbb84e4767b6d6ccf5b7804"
server_key_iv: "8d05d6fc2ffd5dfb1db563a6076aa069"
server_key_version: "encryptionkey@0"
I20260926 15:03:31.897071 18343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:31.931465 18343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:31.936141 18343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:31.936429 18343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:31.937474 18343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:31.938911 18343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:31.938990 18343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:31.939124 18343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:31.939177 18343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:31.975226 18343 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:35349
I20260926 15:03:31.975263 18470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:35349 every 8 connection(s)
I20260926 15:03:31.976512 18343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:03:31.977869 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 18343
I20260926 15:03:31.978042 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:03:31.978480 9709 external_mini_cluster.cc:1428] Setting key 2bc554bf4191aece5c514746e571522e
I20260926 15:03:31.982240 18204 heartbeater.cc:503] Master 127.9.123.126:39341 was elected leader, sending a full tablet report...
Sep 26 15:03:31 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435011, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:03:32.000306 18473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:31.979377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:37087 (local address 127.9.123.126:39341)
0926 15:03:31.985582 (+ 6205us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:31.985587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:31.985603 (+ 16us) server_negotiation.cc:407] Connection header received
0926 15:03:31.985650 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:31.985655 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:31.985729 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:31.985818 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:31.987958 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:31.988915 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:31.990673 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:31.991091 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:31.991197 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:31.995108 (+ 3911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:31.995113 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:31.995117 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:31.995151 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:31.997667 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:31.998379 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:31.998382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:31.998384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:31.998494 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:31.998839 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:31.998842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:31.998844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:31.999115 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:31.999219 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:31.999867 (+ 648us) server_negotiation.cc:299] Negotiation successful
0926 15:03:32.000026 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6088,"thread_start_us":5977,"threads_started":1}
I20260926 15:03:32.001358 18474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:31.980099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.67:37087)
0926 15:03:31.981092 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:31.981185 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:31.983817 (+ 2632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:31.986197 (+ 2380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:31.986212 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:31.987048 (+ 836us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:31.987759 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:31.987780 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:31.989046 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:31.989050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:31.990501 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:31.990516 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:31.991283 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:31.991287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:31.991501 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:31.992225 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:31.992253 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:31.994868 (+ 2615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:31.997799 (+ 2931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:31.997807 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:31.997822 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:31.998287 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:31.998574 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:31.998577 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:31.998579 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:31.998759 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:31.999207 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:31.999213 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:31.999563 (+ 350us) client_negotiation.cc:769] Sending connection context
0926 15:03:31.999793 (+ 230us) client_negotiation.cc:240] Negotiation successful
0926 15:03:32.000194 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":367,"threads_started":1}
I20260926 15:03:32.002777 18471 heartbeater.cc:347] Connected to a master server at 127.9.123.126:39341
I20260926 15:03:32.003072 18471 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:32.003779 18471 heartbeater.cc:511] Master 127.9.123.126:39341 requested a full tablet report, sending...
I20260926 15:03:32.005352 18015 ts_manager.cc:194] Registered new tserver with Master: dd481ee0cb2b4e089e26e74af8ecf5b4 (127.9.123.67:35349)
I20260926 15:03:32.006479 18015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.67@KRBTEST.COM'} at 127.9.123.67:37087
I20260926 15:03:32.006858 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.123.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.123.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.123.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.65.keytab.
Entry for principal oryx/127.9.123.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.65.keytab.
WARNING: no policy specified for oryx/127.9.123.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.123.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.123.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.66.keytab.
Entry for principal oryx/127.9.123.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.66.keytab.
WARNING: no policy specified for oryx/127.9.123.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.123.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.123.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.67.keytab.
Entry for principal oryx/127.9.123.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.67.keytab.
I20260926 15:03:32.139114 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 18076
Sep 26 15:03:32 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.65@KRBTEST.COM
I20260926 15:03:32.154465 18482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:32.140299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51804 (local address 127.9.123.65:41845)
0926 15:03:32.140946 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:32.140953 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:32.140979 (+ 26us) server_negotiation.cc:407] Connection header received
0926 15:03:32.141040 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:32.141046 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:32.141291 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:32.141436 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:32.142729 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:32.143750 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:32.144942 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:32.145586 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:32.145693 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:32.149480 (+ 3787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:32.149493 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:32.149509 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:32.149547 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:32.152095 (+ 2548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:32.152733 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:32.152740 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:32.152742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:32.152875 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:32.153196 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:32.153200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:32.153201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:32.153576 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:32.153793 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:32.153985 (+ 192us) server_negotiation.cc:299] Negotiation successful
0926 15:03:32.154178 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":356,"threads_started":1}
I20260926 15:03:32.155716 18199 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:32.238206 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 18076
I20260926 15:03:32.251410 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 18210
Sep 26 15:03:32 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.66@KRBTEST.COM
I20260926 15:03:32.266597 18484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:32.252307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44294 (local address 127.9.123.66:42873)
0926 15:03:32.252850 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:32.252856 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:32.252886 (+ 30us) server_negotiation.cc:407] Connection header received
0926 15:03:32.252948 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:32.252955 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:32.253235 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:32.253374 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:32.254520 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:32.255476 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:32.256651 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:32.257221 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:32.257310 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:32.261059 (+ 3749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:32.261069 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:32.261083 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:32.261165 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:32.263653 (+ 2488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:32.264347 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:32.264354 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:32.264356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:32.264488 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:32.264907 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:32.264912 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:32.264914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:32.265402 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:32.265662 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:32.266038 (+ 376us) server_negotiation.cc:299] Negotiation successful
0926 15:03:32.266241 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":293,"threads_started":1}
I20260926 15:03:32.267839 18333 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:32.348305 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 18210
I20260926 15:03:32.360733 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 18343
Sep 26 15:03:32 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.67@KRBTEST.COM
I20260926 15:03:32.377403 18486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:32.361574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33864 (local address 127.9.123.67:35349)
0926 15:03:32.362177 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:32.362184 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:32.362213 (+ 29us) server_negotiation.cc:407] Connection header received
0926 15:03:32.362273 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:32.362282 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:32.362584 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:32.362754 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:32.363882 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:32.365445 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:32.366673 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:32.367569 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:32.367700 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:32.371919 (+ 4219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:32.371933 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:32.371947 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:32.371984 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:32.374636 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:32.375314 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:32.375321 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:32.375324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:32.375448 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:32.375805 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:32.375809 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:32.375811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:32.376169 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:32.376476 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:32.376769 (+ 293us) server_negotiation.cc:299] Negotiation successful
0926 15:03:32.377037 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":273,"threads_started":1}
I20260926 15:03:32.378746 18466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:32.459915 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 18343
I20260926 15:03:32.472153 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 17983
I20260926 15:03:32.473341 18045 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:32.573140 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 17983
I20260926 15:03:32.589653 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:39341
--webserver_interface=127.9.123.126
--webserver_port=34469
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:46001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:39341
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:03:32.915480 18489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:32.915781 18489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:32.915869 18489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:32.923094 18489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:03:32.923218 18489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:32.923264 18489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:32.923319 18489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:03:32.923362 18489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:03:32.933420 18489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:46001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:39341
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:39341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=34469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:32.934768 18489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:32.936026 18489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:32.947229 18494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:32.947191 18495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:32.947151 18497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:32.948057 18489 server_base.cc:1034] running on GCE node
I20260926 15:03:32.948884 18489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:32.950349 18489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:32.951548 18489 hybrid_clock.cc:648] HybridClock initialized: now 1790435012951497 us; error 44 us; skew 500 ppm
I20260926 15:03:32.951911 18489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:32 dist-test-slave-7153 krb5kdc[17970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435012, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:32.954878 18489 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:03:32.956332 18489 webserver.cc:460] Webserver started at http://127.9.123.126:34469/ using document root <none> and password file <none>
I20260926 15:03:32.957048 18489 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:32.957206 18489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:32.962917 18489 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260926 15:03:32.965847 18504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:32.967248 18489 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:03:32.967425 18489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "3330df1fe5094f1b8130bf29b070817b"
format_stamp: "Formatted at 2026-09-26 15:03:30 on dist-test-slave-7153"
server_key: "b62e88451852198648991f5c0fe449fe"
server_key_iv: "76cfb1808e33f330ee40300bd89f943e"
server_key_version: "encryptionkey@0"
I20260926 15:03:32.968364 18489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:32.994095 18489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:32.998421 18489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:32.998698 18489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:33.015618 18489 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:39341
I20260926 15:03:33.015635 18555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:39341 every 8 connection(s)
I20260926 15:03:33.017071 18489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:03:33.021406 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 18489
I20260926 15:03:33.022348 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:41845
--local_ip_for_outbound_sockets=127.9.123.65
--tserver_master_addrs=127.9.123.126:39341
--webserver_port=44715
--webserver_interface=127.9.123.65
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:46001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.123.65
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:03:33.025128 18556 sys_catalog.cc:263] Verifying existing consensus state
I20260926 15:03:33.029660 18556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b: Bootstrap starting.
I20260926 15:03:33.048967 18556 log.cc:824] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:33.058112 18556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 15:03:33.058638 18556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b: Bootstrap complete.
I20260926 15:03:33.067577 18556 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3330df1fe5094f1b8130bf29b070817b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39341 } }
I20260926 15:03:33.068480 18556 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3330df1fe5094f1b8130bf29b070817b, State: Initialized, Role: FOLLOWER
I20260926 15:03:33.069141 18556 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3330df1fe5094f1b8130bf29b070817b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39341 } }
I20260926 15:03:33.069379 18556 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:03:33.069502 18556 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:03:33.069680 18556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [term 1 FOLLOWER]: Advancing to term 2
I20260926 15:03:33.073789 18556 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3330df1fe5094f1b8130bf29b070817b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39341 } }
I20260926 15:03:33.074364 18556 leader_election.cc:302] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3330df1fe5094f1b8130bf29b070817b; no voters:
I20260926 15:03:33.074960 18556 leader_election.cc:288] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 15:03:33.075112 18560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [term 2 FOLLOWER]: Leader election won for term 2
I20260926 15:03:33.077250 18556 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:03:33.077381 18560 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [term 2 LEADER]: Becoming Leader. State: Replica: 3330df1fe5094f1b8130bf29b070817b, State: Running, Role: LEADER
I20260926 15:03:33.077965 18560 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3330df1fe5094f1b8130bf29b070817b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39341 } }
I20260926 15:03:33.080534 18561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3330df1fe5094f1b8130bf29b070817b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3330df1fe5094f1b8130bf29b070817b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39341 } } }
I20260926 15:03:33.080938 18561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:33.081463 18562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3330df1fe5094f1b8130bf29b070817b. Latest consensus state: current_term: 2 leader_uuid: "3330df1fe5094f1b8130bf29b070817b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3330df1fe5094f1b8130bf29b070817b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39341 } } }
I20260926 15:03:33.081779 18562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:33.085906 18568 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:03:33.092350 18568 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:03:33.094053 18568 catalog_manager.cc:1259] Loaded cluster ID: 8bcbf891b77648ddb1fbce9e21de5f43
I20260926 15:03:33.094256 18568 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:03:33.098692 18568 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:03:33.100657 18568 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3330df1fe5094f1b8130bf29b070817b: Loaded TSK: 0
I20260926 15:03:33.102738 18568 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 15:03:33.405853 18558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:33.406144 18558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:33.406256 18558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:33.413621 18558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:33.413743 18558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:33.413887 18558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:03:33.423676 18558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:46001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.65.keytab
--principal=oryx/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:41845
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=44715
--tserver_master_addrs=127.9.123.126:39341
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:33.425050 18558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:33.426405 18558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:33.440431 18584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:33.440457 18583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:33.441165 18558 server_base.cc:1034] running on GCE node
W20260926 15:03:33.441331 18586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:33.441996 18558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:33.443037 18558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:33.444276 18558 hybrid_clock.cc:648] HybridClock initialized: now 1790435013444222 us; error 44 us; skew 500 ppm
I20260926 15:03:33.444617 18558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:33 dist-test-slave-7153 krb5kdc[17970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435013, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:33.447964 18558 init.cc:377] Logged in from keytab as oryx/127.9.123.65@KRBTEST.COM (short username oryx)
I20260926 15:03:33.449594 18558 webserver.cc:460] Webserver started at http://127.9.123.65:44715/ using document root <none> and password file <none>
I20260926 15:03:33.450294 18558 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:33.450426 18558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:33.456161 18558 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260926 15:03:33.458990 18593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:33.460407 18558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 15:03:33.460556 18558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "c66fea23b1e24343a9a0a6b4f384c7b8"
format_stamp: "Formatted at 2026-09-26 15:03:30 on dist-test-slave-7153"
server_key: "31670a39ade887b76c9eb02f51f7e4ce"
server_key_iv: "2efadcff7147b6b6320d4e1e3db874b3"
server_key_version: "encryptionkey@0"
I20260926 15:03:33.461531 18558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:33.481634 18558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:33.486521 18558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:33.486788 18558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:33.487871 18558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:33.489257 18558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:33.489349 18558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:33.489441 18558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:33.489521 18558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:33.519275 18558 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:41845
I20260926 15:03:33.519332 18705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:41845 every 8 connection(s)
I20260926 15:03:33.520673 18558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:03:33.527634 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 18558
I20260926 15:03:33.528383 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:42873
--local_ip_for_outbound_sockets=127.9.123.66
--tserver_master_addrs=127.9.123.126:39341
--webserver_port=40259
--webserver_interface=127.9.123.66
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:46001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.123.66
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 15:03:33 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:33 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
W20260926 15:03:33.857460 18710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:33.857766 18710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:33.857885 18710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:33.865255 18710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:33.865388 18710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:33.865554 18710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:03:33.875932 18710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:46001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.66.keytab
--principal=oryx/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:42873
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=40259
--tserver_master_addrs=127.9.123.126:39341
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:33.877404 18710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:33.878856 18710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:33.893363 18717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:33.893409 18719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:33.894131 18716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:33.895056 18710 server_base.cc:1034] running on GCE node
I20260926 15:03:33.895723 18710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:33.896750 18710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:33.898046 18710 hybrid_clock.cc:648] HybridClock initialized: now 1790435013897976 us; error 69 us; skew 500 ppm
I20260926 15:03:33.898413 18710 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:33 dist-test-slave-7153 krb5kdc[17970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435013, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:33.901885 18710 init.cc:377] Logged in from keytab as oryx/127.9.123.66@KRBTEST.COM (short username oryx)
I20260926 15:03:33.903631 18710 webserver.cc:460] Webserver started at http://127.9.123.66:40259/ using document root <none> and password file <none>
I20260926 15:03:33.904392 18710 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:33.904527 18710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:33.910873 18710 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 15:03:33.914389 18726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:33.916288 18710 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260926 15:03:33.916494 18710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "9bed0958116747ada838459e0316cf85"
format_stamp: "Formatted at 2026-09-26 15:03:31 on dist-test-slave-7153"
server_key: "5b541453bb6587492486761126ce65b7"
server_key_iv: "e379968bb30a4c18483aa088ddfc4eda"
server_key_version: "encryptionkey@0"
I20260926 15:03:33.917591 18710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:33.940712 18710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:33.945389 18710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:33.945655 18710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:33.946710 18710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:33.948247 18710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:33.948348 18710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:33.948465 18710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:33.948534 18710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:33.989813 18710 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:42873
I20260926 15:03:33.989861 18838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:42873 every 8 connection(s)
I20260926 15:03:33.991314 18710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:03:33.993062 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 18710
I20260926 15:03:33.993930 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:35349
--local_ip_for_outbound_sockets=127.9.123.67
--tserver_master_addrs=127.9.123.126:39341
--webserver_port=40139
--webserver_interface=127.9.123.67
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:46001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.123.67
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 15:03:34 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:34 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
W20260926 15:03:34.375787 18843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:34.376098 18843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:34.376223 18843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:34.383857 18843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:34.383989 18843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:34.384150 18843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:03:34.394345 18843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:46001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/krb5kdc/oryx_127.9.123.67.keytab
--principal=oryx/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:35349
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=40139
--tserver_master_addrs=127.9.123.126:39341
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:34.395802 18843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:34.397233 18843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:34.412971 18849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:34.413545 18850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:34.415158 18852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:34.416150 18843 server_base.cc:1034] running on GCE node
I20260926 15:03:34.416847 18843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:34.418146 18843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:34.419381 18843 hybrid_clock.cc:648] HybridClock initialized: now 1790435014419315 us; error 55 us; skew 500 ppm
I20260926 15:03:34.419760 18843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:34 dist-test-slave-7153 krb5kdc[17970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435014, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:34.423084 18843 init.cc:377] Logged in from keytab as oryx/127.9.123.67@KRBTEST.COM (short username oryx)
I20260926 15:03:34.426486 18843 webserver.cc:460] Webserver started at http://127.9.123.67:40139/ using document root <none> and password file <none>
I20260926 15:03:34.427239 18843 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:34.427376 18843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:34.433221 18843 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 15:03:34.436188 18859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:34.438081 18843 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:03:34.438259 18843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "dd481ee0cb2b4e089e26e74af8ecf5b4"
format_stamp: "Formatted at 2026-09-26 15:03:31 on dist-test-slave-7153"
server_key: "01ef7e956bbb84e4767b6d6ccf5b7804"
server_key_iv: "8d05d6fc2ffd5dfb1db563a6076aa069"
server_key_version: "encryptionkey@0"
I20260926 15:03:34.439224 18843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:34.454692 18843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:34.459059 18843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:34.459336 18843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:34.460397 18843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:34.461823 18843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:34.461917 18843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:34.462014 18843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:34.462085 18843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:34.500219 18843 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:35349
I20260926 15:03:34.500262 18971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:35349 every 8 connection(s)
I20260926 15:03:34.501811 18843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:03:34.507186 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 18843
Sep 26 15:03:34 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:34.544696 18977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:34.508334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58394 (local address 127.9.123.126:39341)
0926 15:03:34.515714 (+ 7380us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:34.515722 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:34.515748 (+ 26us) server_negotiation.cc:407] Connection header received
0926 15:03:34.515821 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:34.515829 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:34.515919 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:34.516095 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:34.521309 (+ 5214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:34.522875 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:34.526221 (+ 3346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:34.526988 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:34.527111 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:34.532016 (+ 4905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:34.532032 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:34.532061 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:34.532130 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:34.537331 (+ 5201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:34.538578 (+ 1247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:34.538592 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:34.538605 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:34.538957 (+ 352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:34.539548 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:34.539555 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:34.539560 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:34.540379 (+ 819us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:34.540747 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:34.542861 (+ 2114us) server_negotiation.cc:299] Negotiation successful
0926 15:03:34.543456 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7140,"thread_start_us":4928,"threads_started":1}
Sep 26 15:03:34 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
W20260926 15:03:34.667799 18708 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.130s user 0.002s sys 0.010s
W20260926 15:03:34.669157 18709 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:33.523951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:49413 (local address 127.9.123.126:39341)
0926 15:03:33.524872 (+ 921us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:33.524892 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:33.525901 (+ 1009us) server_negotiation.cc:407] Connection header received
0926 15:03:33.528499 (+ 2598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:33.528523 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:33.529057 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:33.529524 (+ 467us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:33.531548 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:33.533370 (+ 1822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:33.535521 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:33.536593 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:33.536759 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:34.668818 (+1132059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.65:49413: BlockingRecv error: recv got EOF from 127.9.123.65:49413 (error 108)
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":393,"threads_started":1}
I20260926 15:03:34.669644 18708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:33.523726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.65:49413)
0926 15:03:33.525684 (+ 1958us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:33.525737 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:33.527763 (+ 2026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:33.529804 (+ 2041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:33.529816 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:33.530584 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:33.531300 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:33.531325 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:33.533549 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:33.533555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:33.535357 (+ 1802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:33.535371 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:33.536847 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:33.536854 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:33.537158 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:33.537865 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:33.537899 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:34.668539 (+1130640us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1667,"thread_start_us":1478,"threads_started":1}
W20260926 15:03:34.670508 18706 heartbeater.cc:644] Failed to heartbeat to 127.9.123.126:39341 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.123.126:39341: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Sep 26 15:03:34 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:34 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:34.680812 18708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:34.671195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.65:40767)
0926 15:03:34.671368 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:34.671388 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:34.671504 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:34.671898 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:34.671903 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:34.672269 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:34.672430 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:34.672438 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:34.673742 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:34.673801 (+ 59us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:34.675460 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:34.675473 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:34.676144 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:34.676150 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:34.676299 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:34.676879 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:34.676896 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:34.680514 (+ 3618us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":51}
W20260926 15:03:34.680989 18709 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:34.671438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:40767 (local address 127.9.123.126:39341)
0926 15:03:34.671629 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:34.671635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:34.671652 (+ 17us) server_negotiation.cc:407] Connection header received
0926 15:03:34.671707 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:34.671710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:34.671765 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:34.671845 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:34.672604 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:34.673601 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:34.675597 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:34.675987 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:34.676090 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:34.680771 (+ 4681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.65:40767: BlockingRecv error: recv got EOF from 127.9.123.65:40767 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Sep 26 15:03:34 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:34 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:34.691494 18708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:34.681882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.65:40003)
0926 15:03:34.682216 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:34.682230 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:34.682344 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:34.682680 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:34.682684 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:34.683021 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:34.683179 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:34.683188 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:34.684230 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:34.684235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:34.685241 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:34.685255 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:34.686157 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:34.686164 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:34.686338 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:34.686986 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:34.687182 (+ 196us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:34.691215 (+ 4033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253}
W20260926 15:03:34.691697 18709 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:34.681955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:40003 (local address 127.9.123.126:39341)
0926 15:03:34.682118 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:34.682122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:34.682302 (+ 180us) server_negotiation.cc:407] Connection header received
0926 15:03:34.682462 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:34.682468 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:34.682554 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:34.682634 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:34.683319 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:34.684089 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:34.685377 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:34.686020 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:34.686134 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:34.691444 (+ 5310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.65:40003: BlockingRecv error: recv got EOF from 127.9.123.65:40003 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260926 15:03:34.692193 18706 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 15:03:35.265234 18842 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.233s user 0.006s sys 0.014s
W20260926 15:03:35.266443 18841 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:33.994528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:37327 (local address 127.9.123.126:39341)
0926 15:03:34.013685 (+ 19157us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:34.013692 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:34.015796 (+ 2104us) server_negotiation.cc:407] Connection header received
0926 15:03:34.021211 (+ 5415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:34.021220 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:34.021363 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:34.021488 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:34.024986 (+ 3498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:34.026473 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:34.029465 (+ 2992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:34.030129 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:34.030295 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:35.266190 (+1235895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.66:37327: BlockingRecv error: recv got EOF from 127.9.123.66:37327 (error 108)
Metrics: {"server-negotiator.queue_time_us":18984,"thread_start_us":19292,"threads_started":1}
I20260926 15:03:35.267079 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:33.995152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.66:37327)
0926 15:03:34.014324 (+ 19172us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:34.014390 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:34.016827 (+ 2437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:34.022605 (+ 5778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:34.022623 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:34.023622 (+ 999us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:34.024663 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:34.024696 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:34.026734 (+ 2038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:34.026741 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:34.029216 (+ 2475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:34.029240 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:34.030458 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:34.030467 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:34.030856 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:34.031826 (+ 970us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:34.031879 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:35.266000 (+1234121us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":18835,"thread_start_us":18600,"threads_started":1}
W20260926 15:03:35.267954 18839 heartbeater.cc:644] Failed to heartbeat to 127.9.123.126:39341 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.123.126:39341: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Sep 26 15:03:35 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:35 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:35.277802 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:35.268675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.66:47949)
0926 15:03:35.268879 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:35.268898 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:35.269021 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:35.269354 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:35.269359 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:35.269743 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:35.269904 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:35.269912 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:35.270922 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:35.270926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:35.271978 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:35.271995 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:35.272752 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:35.272761 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:35.272987 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:35.273798 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:35.273824 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:35.277536 (+ 3712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":68}
W20260926 15:03:35.278216 18841 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:35.268703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:47949 (local address 127.9.123.126:39341)
0926 15:03:35.268869 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:35.268875 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:35.268959 (+ 84us) server_negotiation.cc:407] Connection header received
0926 15:03:35.269168 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:35.269172 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:35.269232 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:35.269315 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:35.270054 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:35.270776 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:35.272116 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:35.272514 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:35.272647 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:35.277873 (+ 5226us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.66:47949: BlockingRecv error: recv got EOF from 127.9.123.66:47949 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 26 15:03:35 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:35 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:35.288875 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:35.278520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.66:58251)
0926 15:03:35.278728 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:35.278753 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:35.278880 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:35.279192 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:35.279197 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:35.279653 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:35.279893 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:35.279906 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:35.280899 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:35.280905 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:35.282703 (+ 1798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:35.282725 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:35.283530 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:35.283538 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:35.283758 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:35.284452 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:35.284480 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:35.288601 (+ 4121us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":64}
W20260926 15:03:35.289124 18841 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:35.278676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:58251 (local address 127.9.123.126:39341)
0926 15:03:35.278870 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:35.278875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:35.278894 (+ 19us) server_negotiation.cc:407] Connection header received
0926 15:03:35.279002 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:35.279005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:35.279070 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:35.279144 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:35.280040 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:35.280758 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:35.282921 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:35.283359 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:35.283469 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:35.288837 (+ 5368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.66:58251: BlockingRecv error: recv got EOF from 127.9.123.66:58251 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260926 15:03:35.289258 18839 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 15:03:35.626135 18975 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.085s user 0.007s sys 0.012s
W20260926 15:03:35.627396 18974 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:34.505341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:43083 (local address 127.9.123.126:39341)
0926 15:03:34.513586 (+ 8245us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:34.513594 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:34.514416 (+ 822us) server_negotiation.cc:407] Connection header received
0926 15:03:34.523909 (+ 9493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:34.523915 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:34.524023 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:34.524144 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:34.533546 (+ 9402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:34.535213 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:34.538464 (+ 3251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:34.539258 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:34.539369 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:35.627092 (+1087723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.67:43083: BlockingRecv error: recv got EOF from 127.9.123.67:43083 (error 108)
Metrics: {"server-negotiator.queue_time_us":8092,"thread_start_us":8009,"threads_started":1}
I20260926 15:03:35.628051 18975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:34.506190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.67:43083)
0926 15:03:34.514093 (+ 7903us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:34.514158 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:34.523485 (+ 9327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:34.531086 (+ 7601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:34.531104 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:34.532077 (+ 973us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:34.533211 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:34.533247 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:34.535527 (+ 2280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:34.535535 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:34.538211 (+ 2676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:34.538234 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:34.539853 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:34.539861 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:34.540291 (+ 430us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:34.541396 (+ 1105us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:34.541446 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:35.626870 (+1085424us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":7498,"thread_start_us":7245,"threads_started":1}
W20260926 15:03:35.628906 18972 heartbeater.cc:644] Failed to heartbeat to 127.9.123.126:39341 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.123.126:39341: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Sep 26 15:03:35 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:35 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:35.645027 18975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:35.629613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.67:54563)
0926 15:03:35.629811 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:35.629832 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:35.629949 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:35.630268 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:35.630273 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:35.630635 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:35.630794 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:35.630803 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:35.631829 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:35.631833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:35.632800 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:35.632813 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:35.633615 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:35.633622 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:35.633774 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:35.641283 (+ 7509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:35.641309 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:35.644797 (+ 3488us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260926 15:03:35.645273 18974 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:35.629699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:54563 (local address 127.9.123.126:39341)
0926 15:03:35.629859 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:35.629864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:35.629883 (+ 19us) server_negotiation.cc:407] Connection header received
0926 15:03:35.630069 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:35.630075 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:35.630141 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:35.630223 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:35.630935 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:35.631689 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:35.632943 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:35.633438 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:35.633563 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:35.645030 (+ 11467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.67:54563: BlockingRecv error: recv got EOF from 127.9.123.67:54563 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 26 15:03:35 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:35 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:35.654443 18975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:35.645771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.67:52975)
0926 15:03:35.645931 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:35.645948 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:35.646052 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:35.646439 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:35.646444 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:35.646777 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:35.646935 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:35.646943 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:35.648011 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:35.648017 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:35.649319 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:35.649337 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:35.650249 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:35.650254 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:35.650385 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:35.650953 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:35.650971 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:35.654235 (+ 3264us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260926 15:03:35.654757 18974 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:35.645864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:52975 (local address 127.9.123.126:39341)
0926 15:03:35.646171 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:35.646175 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:35.646189 (+ 14us) server_negotiation.cc:407] Connection header received
0926 15:03:35.646242 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:35.646246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:35.646313 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:35.646392 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:35.647077 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:35.647844 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:35.649470 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:35.650106 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:35.650203 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:35.654465 (+ 4262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.67:52975: BlockingRecv error: recv got EOF from 127.9.123.67:52975 (error 108)
Metrics: {"server-negotiator.queue_time_us":186}
W20260926 15:03:35.654807 18972 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 26 15:03:35 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:35 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:35.701903 18979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:35.692843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.65:40733)
0926 15:03:35.693403 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:35.693420 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:35.693559 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:35.693896 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:35.693901 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:35.694359 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:35.694515 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:35.694526 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:35.695590 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:35.695594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:35.696568 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:35.696581 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:35.697355 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:35.697362 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:35.697526 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:35.698150 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:35.698169 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:35.701578 (+ 3409us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":326,"threads_started":1}
W20260926 15:03:35.702037 18974 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:35.692981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:40733 (local address 127.9.123.126:39341)
0926 15:03:35.693227 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:35.693234 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:35.693506 (+ 272us) server_negotiation.cc:407] Connection header received
0926 15:03:35.693687 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:35.693693 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:35.693759 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:35.693843 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:35.694699 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:35.695417 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:35.696722 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:35.697166 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:35.697281 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:35.701829 (+ 4548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.65:40733: BlockingRecv error: recv got EOF from 127.9.123.65:40733 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Sep 26 15:03:36 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:36 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:36.299465 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:36.290003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.66:46763)
0926 15:03:36.290551 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:36.290570 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:36.290699 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:36.291106 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:36.291111 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:36.291655 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:36.291857 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:36.291870 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:36.293218 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:36.293222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:36.294208 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:36.294222 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:36.295035 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:36.295040 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:36.295172 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:36.295790 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:36.295810 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:36.299222 (+ 3412us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":304,"threads_started":1}
W20260926 15:03:36.299763 18981 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:36.290138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:46763 (local address 127.9.123.126:39341)
0926 15:03:36.290636 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:36.290641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:36.290662 (+ 21us) server_negotiation.cc:407] Connection header received
0926 15:03:36.290819 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:36.290830 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:36.290912 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:36.291052 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:36.292042 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:36.293061 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:36.294365 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:36.294829 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:36.294932 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:36.299467 (+ 4535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.66:46763: BlockingRecv error: recv got EOF from 127.9.123.66:46763 (error 108)
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":279,"threads_started":1}
Sep 26 15:03:36 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:36 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:36.664234 18982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:36.655446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.67:45937)
0926 15:03:36.656019 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:36.656036 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:36.656169 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:36.656519 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:36.656524 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:36.656935 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:36.657072 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:36.657080 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:36.658147 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:36.658152 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:36.659078 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:36.659090 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:36.659795 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:36.659801 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:36.659957 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:36.660525 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:36.660542 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:36.663915 (+ 3373us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":343,"threads_started":1}
W20260926 15:03:36.664414 18981 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:36.655509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:45937 (local address 127.9.123.126:39341)
0926 15:03:36.655690 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:36.655697 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:36.656267 (+ 570us) server_negotiation.cc:407] Connection header received
0926 15:03:36.656316 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:36.656320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:36.656376 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:36.656448 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:36.657253 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:36.658004 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:36.659212 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:36.659624 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:36.659730 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:36.664202 (+ 4472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.67:45937: BlockingRecv error: recv got EOF from 127.9.123.67:45937 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 26 15:03:36 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:36 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:36.712606 18983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:36.703037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.65:35879)
0926 15:03:36.703788 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:36.703806 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:36.703945 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:36.704298 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:36.704303 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:36.704940 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:36.705174 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:36.705189 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:36.706252 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:36.706256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:36.707242 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:36.707257 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:36.708023 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:36.708028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:36.708165 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:36.708715 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:36.708732 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:36.712319 (+ 3587us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":282,"threads_started":1}
W20260926 15:03:36.712845 18981 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:36.703168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:35879 (local address 127.9.123.126:39341)
0926 15:03:36.703379 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:36.703386 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:36.703918 (+ 532us) server_negotiation.cc:407] Connection header received
0926 15:03:36.704074 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:36.704080 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:36.704152 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:36.704242 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:36.705330 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:36.706107 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:36.707399 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:36.707858 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:36.707957 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:36.712580 (+ 4623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.65:35879: BlockingRecv error: recv got EOF from 127.9.123.65:35879 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Sep 26 15:03:37 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:37 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:37.308986 18984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:37.300374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.66:40935)
0926 15:03:37.300864 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:37.300882 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:37.300995 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:37.301322 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:37.301324 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:37.301705 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:37.301880 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:37.301888 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:37.302831 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:37.302840 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:37.303799 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:37.303813 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:37.304531 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:37.304536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:37.304674 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:37.305328 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:37.305346 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:37.308656 (+ 3310us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":246,"threads_started":1}
W20260926 15:03:37.309136 18985 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:37.300523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:40935 (local address 127.9.123.126:39341)
0926 15:03:37.300950 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:37.300954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:37.300972 (+ 18us) server_negotiation.cc:407] Connection header received
0926 15:03:37.301125 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:37.301130 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:37.301195 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:37.301284 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:37.301989 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:37.302706 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:37.303947 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:37.304376 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:37.304474 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:37.308875 (+ 4401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.66:40935: BlockingRecv error: recv got EOF from 127.9.123.66:40935 (error 108)
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":233,"threads_started":1}
Sep 26 15:03:37 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:37 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:37.674011 18986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:37.665211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.67:50109)
0926 15:03:37.665701 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:37.665716 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:37.665844 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:37.666189 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:37.666194 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:37.666641 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:37.666821 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:37.666830 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:37.667838 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:37.667844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:37.668775 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:37.668788 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:37.669498 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:37.669504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:37.669679 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:37.670259 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:37.670276 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:37.673745 (+ 3469us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":270,"threads_started":1}
W20260926 15:03:37.674125 18985 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:37.665380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:50109 (local address 127.9.123.126:39341)
0926 15:03:37.665612 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:37.665618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:37.665789 (+ 171us) server_negotiation.cc:407] Connection header received
0926 15:03:37.665972 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:37.665977 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:37.666047 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:37.666125 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:37.666966 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:37.667712 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:37.668920 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:37.669354 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:37.669472 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:37.673944 (+ 4472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.67:50109: BlockingRecv error: recv got EOF from 127.9.123.67:50109 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 26 15:03:37 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:37 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:37.722342 18987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:37.713554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.65:47421)
0926 15:03:37.714042 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:37.714058 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:37.714228 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:37.714540 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:37.714545 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:37.714925 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:37.715086 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:37.715095 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:37.716113 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:37.716117 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:37.717039 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:37.717050 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:37.717781 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:37.717787 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:37.717947 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:37.718538 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:37.718555 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:37.722032 (+ 3477us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":236,"threads_started":1}
W20260926 15:03:37.722481 18985 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:37.713700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:47421 (local address 127.9.123.126:39341)
0926 15:03:37.713900 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:37.713906 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:37.714134 (+ 228us) server_negotiation.cc:407] Connection header received
0926 15:03:37.714345 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:37.714349 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:37.714419 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:37.714499 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:37.715234 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:37.715972 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:37.717219 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:37.717635 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:37.717754 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:37.722220 (+ 4466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.65:47421: BlockingRecv error: recv got EOF from 127.9.123.65:47421 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 26 15:03:38 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:38 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:38.318574 18988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:38.310243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.66:53543)
0926 15:03:38.310742 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:38.310763 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:38.310874 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:38.311192 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:38.311195 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:38.311523 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:38.311684 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:38.311693 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:38.312737 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:38.312741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:38.313674 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:38.313686 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:38.314499 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:38.314507 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:38.314680 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:38.315286 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:38.315305 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:38.318346 (+ 3041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":245,"threads_started":1}
W20260926 15:03:38.318821 18989 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:38.310376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:53543 (local address 127.9.123.126:39341)
0926 15:03:38.310889 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:38.310893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:38.310913 (+ 20us) server_negotiation.cc:407] Connection header received
0926 15:03:38.310978 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:38.310981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:38.311049 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:38.311147 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:38.311836 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:38.312587 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:38.313792 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:38.314321 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:38.314450 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:38.318552 (+ 4102us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.66:53543: BlockingRecv error: recv got EOF from 127.9.123.66:53543 (error 108)
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":280,"threads_started":1}
Sep 26 15:03:38 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:38 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:38.682765 18990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:38.674814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.67:39095)
0926 15:03:38.675295 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:38.675310 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:38.675414 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:38.675662 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:38.675664 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:38.675995 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:38.676149 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:38.676156 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:38.677185 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:38.677190 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:38.678069 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:38.678079 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:38.678746 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:38.678750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:38.678864 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:38.679378 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:38.679394 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:38.682426 (+ 3032us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":241,"threads_started":1}
W20260926 15:03:38.682824 18989 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:38.674896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:39095 (local address 127.9.123.126:39341)
0926 15:03:38.675053 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:38.675057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:38.675372 (+ 315us) server_negotiation.cc:407] Connection header received
0926 15:03:38.675497 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:38.675500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:38.675558 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:38.675634 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:38.676270 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:38.677018 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:38.678180 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:38.678617 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:38.678711 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:38.682621 (+ 3910us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.67:39095: BlockingRecv error: recv got EOF from 127.9.123.67:39095 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 26 15:03:38 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:38 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:38.731797 18991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:38.723215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.65:45167)
0926 15:03:38.723793 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:38.723813 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:38.723940 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:38.724255 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:38.724260 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:38.724634 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:38.724800 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:38.724809 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:38.725815 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:38.725820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:38.726802 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:38.726813 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:38.727489 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:38.727495 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:38.727666 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:38.728222 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:38.728238 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:38.731521 (+ 3283us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":319,"threads_started":1}
W20260926 15:03:38.732052 18989 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:38.723291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:45167 (local address 127.9.123.126:39341)
0926 15:03:38.723517 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:38.723523 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:38.723890 (+ 367us) server_negotiation.cc:407] Connection header received
0926 15:03:38.724050 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:38.724056 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:38.724129 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:38.724217 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:38.724936 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:38.725696 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:38.726935 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:38.727345 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:38.727448 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:38.731783 (+ 4335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.65:45167: BlockingRecv error: recv got EOF from 127.9.123.65:45167 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Sep 26 15:03:39 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:39 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:39.328613 18992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:39.319677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.66:51135)
0926 15:03:39.320183 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:39.320200 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:39.320313 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:39.320700 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:39.320704 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:39.321048 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:39.321268 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:39.321277 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:39.322362 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:39.322366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:39.323443 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:39.323456 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:39.324174 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:39.324179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:39.324313 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:39.324926 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:39.324966 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:39.328363 (+ 3397us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":262,"threads_started":1}
W20260926 15:03:39.328830 18993 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:39.319734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:51135 (local address 127.9.123.126:39341)
0926 15:03:39.320217 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:39.320221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:39.320256 (+ 35us) server_negotiation.cc:407] Connection header received
0926 15:03:39.320470 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:39.320475 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:39.320538 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:39.320643 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:39.321399 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:39.322225 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:39.323598 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:39.324018 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:39.324118 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:39.328598 (+ 4480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.66:51135: BlockingRecv error: recv got EOF from 127.9.123.66:51135 (error 108)
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":248,"threads_started":1}
Sep 26 15:03:39 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:39 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:39.692540 18994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:39.683844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.67:44803)
0926 15:03:39.684371 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:39.684388 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:39.684508 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:39.684777 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:39.684780 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:39.685188 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:39.685359 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:39.685369 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:39.686425 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:39.686429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:39.687457 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:39.687470 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:39.688142 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:39.688146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:39.688273 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:39.688845 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:39.688863 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:39.692235 (+ 3372us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":294,"threads_started":1}
W20260926 15:03:39.692687 18993 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:39.683930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:44803 (local address 127.9.123.126:39341)
0926 15:03:39.684105 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:39.684109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:39.684452 (+ 343us) server_negotiation.cc:407] Connection header received
0926 15:03:39.684601 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:39.684604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:39.684664 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:39.684749 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:39.685467 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:39.686284 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:39.687598 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:39.688004 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:39.688092 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:39.692425 (+ 4333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.67:44803: BlockingRecv error: recv got EOF from 127.9.123.67:44803 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 26 15:03:39 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:39 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:39.740598 18995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:39.732675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.65:59041)
0926 15:03:39.733201 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:39.733218 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:39.733329 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:39.733600 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:39.733603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:39.734003 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:39.734147 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:39.734155 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:39.735080 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:39.735084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:39.736043 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:39.736054 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:39.736655 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:39.736659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:39.736775 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:39.737335 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:39.737351 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:39.740376 (+ 3025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":274,"threads_started":1}
W20260926 15:03:39.740804 18993 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:39.732807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:59041 (local address 127.9.123.126:39341)
0926 15:03:39.733036 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:39.733041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:39.733288 (+ 247us) server_negotiation.cc:407] Connection header received
0926 15:03:39.733417 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:39.733420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:39.733490 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:39.733568 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:39.734256 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:39.734952 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:39.736151 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:39.736546 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:39.736631 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:39.740571 (+ 3940us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.65:59041: BlockingRecv error: recv got EOF from 127.9.123.65:59041 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 26 15:03:40 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:40 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:40.338428 18996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:40.329800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.66:43325)
0926 15:03:40.330368 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:40.330392 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:40.330533 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:40.330987 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:40.330992 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:40.331405 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:40.331588 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:40.331597 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:40.332562 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:40.332565 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:40.333544 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:40.333558 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:40.334187 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:40.334191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:40.334304 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:40.334836 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:40.334853 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:40.338200 (+ 3347us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":256,"threads_started":1}
W20260926 15:03:40.338613 18997 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:40.330002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:43325 (local address 127.9.123.126:39341)
0926 15:03:40.330684 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:40.330689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:40.330717 (+ 28us) server_negotiation.cc:407] Connection header received
0926 15:03:40.330766 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:40.330769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:40.330831 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:40.330910 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:40.331737 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:40.332452 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:40.333662 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:40.334067 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:40.334154 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:40.338399 (+ 4245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.66:43325: BlockingRecv error: recv got EOF from 127.9.123.66:43325 (error 108)
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":459,"threads_started":1}
Sep 26 15:03:40 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:40 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:40.709569 18998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:40.693562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.67:46143)
0926 15:03:40.694088 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:40.694104 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:40.694242 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:40.694636 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:40.694641 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:40.695074 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:40.695223 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:40.695232 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:40.696236 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:40.696242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:40.697342 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:40.697359 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:40.698154 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:40.698160 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:40.698320 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:40.705537 (+ 7217us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:40.705563 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:40.709294 (+ 3731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":281,"threads_started":1}
W20260926 15:03:40.709786 18997 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:40.693719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:46143 (local address 127.9.123.126:39341)
0926 15:03:40.693949 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:40.693954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:40.694192 (+ 238us) server_negotiation.cc:407] Connection header received
0926 15:03:40.694363 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:40.694367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:40.694430 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:40.694523 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:40.695373 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:40.696092 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:40.697517 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:40.697972 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:40.698087 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:40.709567 (+ 11480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.67:46143: BlockingRecv error: recv got EOF from 127.9.123.67:46143 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Sep 26 15:03:40 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:40 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:40.750551 18999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:40.741458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.65:59287)
0926 15:03:40.741937 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:40.741954 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:40.742069 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:40.742385 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:40.742389 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:40.742777 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:40.742941 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:40.742950 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:40.744035 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:40.744040 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:40.745043 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:40.745057 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:40.745945 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:40.745950 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:40.746093 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:40.746705 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:40.746723 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:40.750232 (+ 3509us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":251,"threads_started":1}
W20260926 15:03:40.750778 18997 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:40.741541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:59287 (local address 127.9.123.126:39341)
0926 15:03:40.741788 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:40.741793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:40.742027 (+ 234us) server_negotiation.cc:407] Connection header received
0926 15:03:40.742173 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:40.742177 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:40.742243 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:40.742335 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:40.743087 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:40.743873 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:40.745243 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:40.745737 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:40.745851 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:40.750463 (+ 4612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.65:59287: BlockingRecv error: recv got EOF from 127.9.123.65:59287 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 26 15:03:41 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:41 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:41.348011 19000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:41.339329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.66:53257)
0926 15:03:41.339813 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:41.339831 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:41.339947 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:41.340235 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:41.340238 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:41.340626 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:41.340804 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:41.340812 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:41.341932 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:41.341936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:41.342940 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:41.342953 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:41.343601 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:41.343606 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:41.343765 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:41.344352 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:41.344370 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:41.347692 (+ 3322us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":258,"threads_started":1}
W20260926 15:03:41.348244 19001 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:41.339374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:53257 (local address 127.9.123.126:39341)
0926 15:03:41.339815 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:41.339819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:41.339978 (+ 159us) server_negotiation.cc:407] Connection header received
0926 15:03:41.340034 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:41.340038 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:41.340101 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:41.340194 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:41.340954 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:41.341793 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:41.343047 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:41.343465 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:41.343557 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:41.347994 (+ 4437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.66:53257: BlockingRecv error: recv got EOF from 127.9.123.66:53257 (error 108)
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":259,"threads_started":1}
Sep 26 15:03:41 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:41 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:41.719784 19002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:41.710669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.67:39395)
0926 15:03:41.711134 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:41.711151 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:41.711276 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:41.711648 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:41.711652 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:41.712052 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:41.712221 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:41.712231 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:41.713344 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:41.713348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:41.714295 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:41.714309 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:41.714983 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:41.714988 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:41.715117 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:41.715941 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:41.716076 (+ 135us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:41.719548 (+ 3472us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":237,"threads_started":1}
W20260926 15:03:41.719991 19001 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:41.710718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:39395 (local address 127.9.123.126:39341)
0926 15:03:41.710958 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:41.710964 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:41.711247 (+ 283us) server_negotiation.cc:407] Connection header received
0926 15:03:41.711412 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:41.711417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:41.711482 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:41.711570 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:41.712375 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:41.713191 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:41.714435 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:41.714835 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:41.714931 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:41.719779 (+ 4848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.67:39395: BlockingRecv error: recv got EOF from 127.9.123.67:39395 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Sep 26 15:03:41 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:41 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:41.760893 19003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:41.751737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.65:41473)
0926 15:03:41.752342 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:41.752359 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:41.752478 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:41.752782 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:41.752786 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:41.753223 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:41.753401 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:41.753410 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:41.754579 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:41.754592 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:41.755748 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:41.755761 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:41.756528 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:41.756533 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:41.756684 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:41.757262 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:41.757281 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:41.760664 (+ 3383us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":281,"threads_started":1}
W20260926 15:03:41.761147 19001 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:41.751844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:41473 (local address 127.9.123.126:39341)
0926 15:03:41.752018 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:41.752022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:41.752428 (+ 406us) server_negotiation.cc:407] Connection header received
0926 15:03:41.752570 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:41.752574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:41.752650 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:41.752741 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:41.753519 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:41.754390 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:41.755874 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:41.756392 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:41.756485 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:41.760852 (+ 4367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.65:41473: BlockingRecv error: recv got EOF from 127.9.123.65:41473 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 26 15:03:42 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:42 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:42.358392 19004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:42.349253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.66:53305)
0926 15:03:42.349813 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:42.349830 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:42.349948 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:42.350365 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:42.350369 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:42.350789 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:42.351003 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:42.351017 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:42.352282 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:42.352286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:42.353330 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:42.353343 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:42.354026 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:42.354031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:42.354143 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:42.354687 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:42.354704 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:42.358113 (+ 3409us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":259,"threads_started":1}
W20260926 15:03:42.358582 19005 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:42.349341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:53305 (local address 127.9.123.126:39341)
0926 15:03:42.349813 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:42.349818 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:42.349917 (+ 99us) server_negotiation.cc:407] Connection header received
0926 15:03:42.350061 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:42.350066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:42.350176 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:42.350296 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:42.351176 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:42.352149 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:42.353466 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:42.353892 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:42.353986 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:42.358342 (+ 4356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.66:53305: BlockingRecv error: recv got EOF from 127.9.123.66:53305 (error 108)
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":256,"threads_started":1}
Sep 26 15:03:42 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:42 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:42.730407 19006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:42.720813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.67:53175)
0926 15:03:42.721524 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:42.721540 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:42.721678 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:42.722042 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:42.722046 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:42.722510 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:42.722729 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:42.722741 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:42.723930 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:42.723935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:42.724997 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:42.725011 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:42.725754 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:42.725759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:42.725900 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:42.726492 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:42.726511 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:42.730154 (+ 3643us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":368,"threads_started":1}
W20260926 15:03:42.730576 19005 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:42.721004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:53175 (local address 127.9.123.126:39341)
0926 15:03:42.721215 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:42.721219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:42.721666 (+ 447us) server_negotiation.cc:407] Connection header received
0926 15:03:42.721828 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:42.721832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:42.721898 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:42.722005 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:42.722866 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:42.723780 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:42.725158 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:42.725580 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:42.725686 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:42.730381 (+ 4695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.67:53175: BlockingRecv error: recv got EOF from 127.9.123.67:53175 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 26 15:03:42 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:42 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:42.770104 19007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:42.761770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.65:40081)
0926 15:03:42.762283 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:42.762300 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:42.762410 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:42.762741 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:42.762745 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:42.763123 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:42.763290 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:42.763298 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:42.764264 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:42.764268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:42.765270 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:42.765285 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:42.765947 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:42.765951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:42.766062 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:42.766689 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:42.766708 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:42.769786 (+ 3078us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":280,"threads_started":1}
W20260926 15:03:42.770233 19005 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:42.761960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:40081 (local address 127.9.123.126:39341)
0926 15:03:42.762175 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:42.762181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:42.762372 (+ 191us) server_negotiation.cc:407] Connection header received
0926 15:03:42.762500 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:42.762503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:42.762572 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:42.762658 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:42.763398 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:42.764116 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:42.765402 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:42.765814 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:42.765900 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:42.770000 (+ 4100us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.65:40081: BlockingRecv error: recv got EOF from 127.9.123.65:40081 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 26 15:03:43 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:43 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:43.368132 19008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:43.359260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.66:57753)
0926 15:03:43.359756 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:43.359772 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:43.359872 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:43.360226 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:43.360229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:43.360639 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:43.360802 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:43.360811 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:43.362082 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:43.362088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:43.363019 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:43.363031 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:43.363784 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:43.363789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:43.363910 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:43.364460 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:43.364479 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:43.367809 (+ 3330us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":248,"threads_started":1}
W20260926 15:03:43.368255 19009 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:43.359372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:57753 (local address 127.9.123.126:39341)
0926 15:03:43.359827 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:43.359831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:43.359853 (+ 22us) server_negotiation.cc:407] Connection header received
0926 15:03:43.359966 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:43.359970 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:43.360028 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:43.360188 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:43.360961 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:43.361925 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:43.363148 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:43.363630 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:43.363718 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:43.368016 (+ 4298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.66:57753: BlockingRecv error: recv got EOF from 127.9.123.66:57753 (error 108)
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":288,"threads_started":1}
Sep 26 15:03:43 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:43 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:43.740020 19010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:43.731421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.67:35539)
0926 15:03:43.731971 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:43.731987 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:43.732097 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:43.732405 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:43.732408 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:43.732825 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:43.732984 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:43.732993 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:43.734039 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:43.734043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:43.735018 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:43.735031 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:43.735738 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:43.735742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:43.735853 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:43.736436 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:43.736453 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:43.739772 (+ 3319us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":320,"threads_started":1}
W20260926 15:03:43.740178 19009 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:43.731532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:35539 (local address 127.9.123.126:39341)
0926 15:03:43.731779 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:43.731785 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:43.732060 (+ 275us) server_negotiation.cc:407] Connection header received
0926 15:03:43.732182 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:43.732186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:43.732246 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:43.732322 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:43.733160 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:43.733905 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:43.735178 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:43.735586 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:43.735697 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:43.739970 (+ 4273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.67:35539: BlockingRecv error: recv got EOF from 127.9.123.67:35539 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 26 15:03:43 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:43 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:43.779927 19011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:43.771087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.65:49123)
0926 15:03:43.771580 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:43.771595 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:43.771769 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:43.772080 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:43.772086 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:43.772471 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:43.772634 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:43.772644 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:43.773708 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:43.773714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:43.774628 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:43.774639 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:43.775337 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:43.775343 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:43.775506 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:43.776072 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:43.776089 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:43.779679 (+ 3590us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":270,"threads_started":1}
W20260926 15:03:43.780148 19009 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:43.771168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:49123 (local address 127.9.123.126:39341)
0926 15:03:43.771367 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:43.771373 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:43.771673 (+ 300us) server_negotiation.cc:407] Connection header received
0926 15:03:43.771896 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:43.771900 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:43.771960 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:43.772033 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:43.772771 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:43.773584 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:43.774770 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:43.775195 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:43.775299 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:43.779909 (+ 4610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.65:49123: BlockingRecv error: recv got EOF from 127.9.123.65:49123 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 26 15:03:44 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:44 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:44.378409 19012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:44.369155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.66:45141)
0926 15:03:44.369694 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:44.369714 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:44.369844 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:44.370308 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:44.370311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:44.370801 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:44.370987 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:44.370999 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:44.372044 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:44.372049 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:44.373061 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:44.373075 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:44.373851 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:44.373856 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:44.373987 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:44.374543 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:44.374561 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:44.378119 (+ 3558us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":254,"threads_started":1}
W20260926 15:03:44.378762 19013 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:44.369467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:45141 (local address 127.9.123.126:39341)
0926 15:03:44.370005 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:44.370010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:44.370028 (+ 18us) server_negotiation.cc:407] Connection header received
0926 15:03:44.370075 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:44.370080 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:44.370152 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:44.370277 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:44.371137 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:44.371919 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:44.373241 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:44.373703 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:44.373797 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:44.378439 (+ 4642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.66:45141: BlockingRecv error: recv got EOF from 127.9.123.66:45141 (error 108)
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":316,"threads_started":1}
Sep 26 15:03:44 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:44 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:44.750705 19014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:44.741197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.67:36223)
0926 15:03:44.741738 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:44.741754 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:44.741872 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:44.742199 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:44.742202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:44.742569 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:44.742765 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:44.742778 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:44.743855 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:44.743861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:44.745236 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:44.745252 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:44.746010 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:44.746017 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:44.746197 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:44.746779 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:44.746798 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:44.750455 (+ 3657us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":251,"threads_started":1}
W20260926 15:03:44.750938 19013 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:44.741238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:36223 (local address 127.9.123.126:39341)
0926 15:03:44.741420 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:44.741426 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:44.741861 (+ 435us) server_negotiation.cc:407] Connection header received
0926 15:03:44.741975 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:44.741979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:44.742044 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:44.742153 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:44.742897 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:44.743650 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:44.745361 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:44.745858 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:44.745971 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:44.750707 (+ 4736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.67:36223: BlockingRecv error: recv got EOF from 127.9.123.67:36223 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 26 15:03:44 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:44 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:44.789780 19015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:44.780991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.65:59559)
0926 15:03:44.781717 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:44.781732 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:44.781865 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:44.782231 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:44.782236 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:44.782600 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:44.782761 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:44.782770 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:44.783749 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:44.783755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:44.784713 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:44.784725 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:44.785488 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:44.785494 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:44.785642 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:44.786237 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:44.786256 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:44.789434 (+ 3178us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":499,"threads_started":1}
W20260926 15:03:44.789921 19013 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:44.781207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:59559 (local address 127.9.123.126:39341)
0926 15:03:44.781417 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:44.781424 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:44.781821 (+ 397us) server_negotiation.cc:407] Connection header received
0926 15:03:44.781983 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:44.781988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:44.782068 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:44.782151 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:44.782908 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:44.783619 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:44.784852 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:44.785320 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:44.785427 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:44.789683 (+ 4256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.65:59559: BlockingRecv error: recv got EOF from 127.9.123.65:59559 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 26 15:03:45 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:45 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:45.388949 19016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:45.379459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.66:47659)
0926 15:03:45.379991 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:45.380012 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:45.380133 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:45.380617 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:45.380621 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:45.381028 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:45.381233 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:45.381243 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:45.382279 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:45.382283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:45.383384 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:45.383397 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:45.384240 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:45.384245 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:45.384386 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:45.385083 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:45.385135 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:45.388703 (+ 3568us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":239,"threads_started":1}
W20260926 15:03:45.389226 19017 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:45.379614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:47659 (local address 127.9.123.126:39341)
0926 15:03:45.380141 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:45.380145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:45.380163 (+ 18us) server_negotiation.cc:407] Connection header received
0926 15:03:45.380358 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:45.380363 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:45.380445 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:45.380541 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:45.381379 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:45.382139 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:45.383495 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:45.384061 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:45.384184 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:45.388943 (+ 4759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.66:47659: BlockingRecv error: recv got EOF from 127.9.123.66:47659 (error 108)
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":247,"threads_started":1}
Sep 26 15:03:45 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:45 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:45.767892 19018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:45.751647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.67:35331)
0926 15:03:45.752124 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:45.752144 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:45.752294 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:45.752658 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:45.752664 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:45.753024 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:45.753228 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:45.753239 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:45.754310 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:45.754315 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:45.755687 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:45.755705 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:45.756432 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:45.756436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:45.756551 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:45.763971 (+ 7420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:45.763994 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:45.767549 (+ 3555us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":222,"threads_started":1}
W20260926 15:03:45.767987 19017 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:45.751880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:35331 (local address 127.9.123.126:39341)
0926 15:03:45.752133 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:45.752139 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:45.752229 (+ 90us) server_negotiation.cc:407] Connection header received
0926 15:03:45.752414 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:45.752420 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:45.752480 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:45.752563 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:45.753385 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:45.754166 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:45.755853 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:45.756289 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:45.756389 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:45.767765 (+ 11376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.67:35331: BlockingRecv error: recv got EOF from 127.9.123.67:35331 (error 108)
Metrics: {"server-negotiator.queue_time_us":133}
Sep 26 15:03:45 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:45 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:45.799427 19019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:45.790937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.65:36755)
0926 15:03:45.791390 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:45.791406 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:45.791538 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:45.791949 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:45.791953 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:45.792361 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:45.792575 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:45.792585 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:45.793608 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:45.793611 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:45.794523 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:45.794534 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:45.795151 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:45.795155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:45.795311 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:45.795922 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:45.795940 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:45.799181 (+ 3241us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":229,"threads_started":1}
W20260926 15:03:45.799619 19017 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:45.791172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:36755 (local address 127.9.123.126:39341)
0926 15:03:45.791352 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:45.791356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:45.791492 (+ 136us) server_negotiation.cc:407] Connection header received
0926 15:03:45.791698 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:45.791704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:45.791793 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:45.791885 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:45.792686 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:45.793475 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:45.794629 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:45.795024 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:45.795111 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:45.799399 (+ 4288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.65:36755: BlockingRecv error: recv got EOF from 127.9.123.65:36755 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Sep 26 15:03:46 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:46 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:46.398430 19020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:46.389932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.66:45177)
0926 15:03:46.390422 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:46.390440 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:46.390559 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:46.390924 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:46.390929 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:46.391318 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:46.391488 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:46.391497 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:46.392467 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:46.392470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:46.393399 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:46.393410 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:46.394092 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:46.394097 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:46.394223 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:46.394786 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:46.394804 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:46.398198 (+ 3394us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":260,"threads_started":1}
W20260926 15:03:46.398684 19021 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:46.390021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:45177 (local address 127.9.123.126:39341)
0926 15:03:46.390447 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:46.390453 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:46.390494 (+ 41us) server_negotiation.cc:407] Connection header received
0926 15:03:46.390665 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:46.390668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:46.390735 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:46.390872 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:46.391630 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:46.392357 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:46.393536 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:46.393958 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:46.394061 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:46.398426 (+ 4365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.66:45177: BlockingRecv error: recv got EOF from 127.9.123.66:45177 (error 108)
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":217,"threads_started":1}
Sep 26 15:03:46 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:46 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:46.778182 19022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:46.768884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.67:49323)
0926 15:03:46.769397 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:46.769415 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:46.769530 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:46.769887 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:46.769891 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:46.770414 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:46.770635 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:46.770645 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:46.771753 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:46.771757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:46.772706 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:46.772720 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:46.773675 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:46.773681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:46.773860 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:46.774528 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:46.774546 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:46.777967 (+ 3421us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":303,"threads_started":1}
W20260926 15:03:46.778353 19021 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:46.769032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:49323 (local address 127.9.123.126:39341)
0926 15:03:46.769251 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:46.769257 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:46.769473 (+ 216us) server_negotiation.cc:407] Connection header received
0926 15:03:46.769640 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:46.769644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:46.769710 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:46.769797 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:46.770814 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:46.771595 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:46.772900 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:46.773488 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:46.773598 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:46.778170 (+ 4572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.67:49323: BlockingRecv error: recv got EOF from 127.9.123.67:49323 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 26 15:03:46 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:46 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:46.809352 19023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:46.800432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.65:33009)
0926 15:03:46.800986 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:46.801003 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:46.801193 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:46.801526 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:46.801531 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:46.801941 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:46.802116 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:46.802126 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:46.803126 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:46.803132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:46.804055 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:46.804066 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:46.804798 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:46.804804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:46.805002 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:46.805647 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:46.805669 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:46.808971 (+ 3302us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":302,"threads_started":1}
W20260926 15:03:46.809522 19021 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:46.800503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:33009 (local address 127.9.123.126:39341)
0926 15:03:46.800693 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:46.800699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:46.801084 (+ 385us) server_negotiation.cc:407] Connection header received
0926 15:03:46.801310 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:46.801314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:46.801387 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:46.801465 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:46.802260 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:46.802983 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:46.804211 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:46.804660 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:46.804767 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:46.809248 (+ 4481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.65:33009: BlockingRecv error: recv got EOF from 127.9.123.65:33009 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 26 15:03:47 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:47 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:47.410209 19024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:47.399630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.66:49729)
0926 15:03:47.400305 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:47.400323 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:47.400475 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:47.400890 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:47.400894 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:47.401399 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:47.401595 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:47.401606 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:47.402868 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:47.402873 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:47.403870 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:47.403886 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:47.404888 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:47.404896 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:47.405053 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:47.405975 (+ 922us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:47.405996 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:47.409844 (+ 3848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":416,"threads_started":1}
W20260926 15:03:47.410392 19025 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:47.399787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:49729 (local address 127.9.123.126:39341)
0926 15:03:47.400275 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:47.400280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:47.400443 (+ 163us) server_negotiation.cc:407] Connection header received
0926 15:03:47.400596 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:47.400600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:47.400690 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:47.400804 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:47.401819 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:47.402663 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:47.404063 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:47.404593 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:47.404724 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:47.410133 (+ 5409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.66:49729: BlockingRecv error: recv got EOF from 127.9.123.66:49729 (error 108)
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":224,"threads_started":1}
Sep 26 15:03:47 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:47 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:47.787824 19026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:47.779052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.67:52035)
0926 15:03:47.779635 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:47.779651 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:47.779834 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:47.780160 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:47.780165 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:47.780675 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:47.780857 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:47.780866 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:47.781921 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:47.781927 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:47.782835 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:47.782846 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:47.783522 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:47.783528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:47.783689 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:47.784275 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:47.784293 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:47.787557 (+ 3264us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":311,"threads_started":1}
W20260926 15:03:47.788050 19025 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:47.779134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:52035 (local address 127.9.123.126:39341)
0926 15:03:47.779349 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:47.779356 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:47.779751 (+ 395us) server_negotiation.cc:407] Connection header received
0926 15:03:47.779958 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:47.779963 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:47.780029 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:47.780111 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:47.780995 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:47.781793 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:47.782968 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:47.783362 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:47.783466 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:47.787817 (+ 4351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.67:52035: BlockingRecv error: recv got EOF from 127.9.123.67:52035 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 26 15:03:47 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:47 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:47.819412 19027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:47.810446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.65:37913)
0926 15:03:47.810929 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:47.810945 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:47.811069 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:47.811413 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:47.811416 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:47.811838 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:47.811985 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:47.811993 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:47.813168 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:47.813172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:47.814097 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:47.814111 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:47.814816 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:47.814820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:47.814949 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:47.815554 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:47.815572 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:47.819165 (+ 3593us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":241,"threads_started":1}
W20260926 15:03:47.819615 19025 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:47.810604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:37913 (local address 127.9.123.126:39341)
0926 15:03:47.810842 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:47.810848 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:47.811022 (+ 174us) server_negotiation.cc:407] Connection header received
0926 15:03:47.811190 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:47.811194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:47.811273 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:47.811353 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:47.812150 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:47.812987 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:47.814245 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:47.814653 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:47.814748 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:47.819383 (+ 4635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.65:37913: BlockingRecv error: recv got EOF from 127.9.123.65:37913 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Sep 26 15:03:48 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:48 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:48.425180 19028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:48.411423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.66:54597)
0926 15:03:48.412095 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:48.412118 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:48.412278 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:48.412917 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:48.412955 (+ 38us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:48.413438 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:48.413598 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:48.413607 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:48.415012 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:48.415018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:48.416441 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:48.416457 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:48.417420 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:48.417427 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:48.417617 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:48.418251 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:48.418271 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:48.424808 (+ 6537us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":359,"threads_started":1}
W20260926 15:03:48.425364 19029 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:48.411802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:54597 (local address 127.9.123.126:39341)
0926 15:03:48.412538 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:48.412544 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:48.412564 (+ 20us) server_negotiation.cc:407] Connection header received
0926 15:03:48.412625 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:48.412631 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:48.412705 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:48.412819 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:48.413757 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:48.414849 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:48.416617 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:48.417265 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:48.417377 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:48.425073 (+ 7696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.66:54597: BlockingRecv error: recv got EOF from 127.9.123.66:54597 (error 108)
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":518,"threads_started":1}
Sep 26 15:03:48 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:48 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.67@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:48.798302 19030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:48.788723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.67:34631)
0926 15:03:48.789247 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:48.789264 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:48.789422 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:48.790024 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:48.790029 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:48.790456 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:48.790601 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:48.790611 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:48.791807 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:48.791811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:48.792744 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:48.792756 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:48.793610 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:48.793617 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:48.793799 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:48.794429 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:48.794445 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:48.797979 (+ 3534us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":275,"threads_started":1}
W20260926 15:03:48.798448 19029 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:48.788823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:34631 (local address 127.9.123.126:39341)
0926 15:03:48.789065 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:48.789071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:48.789349 (+ 278us) server_negotiation.cc:407] Connection header received
0926 15:03:48.789657 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:48.789661 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:48.789846 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:48.789966 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:48.790760 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:48.791618 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:48.792906 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:48.793379 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:48.793589 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:48.798219 (+ 4630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.67:34631: BlockingRecv error: recv got EOF from 127.9.123.67:34631 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Sep 26 15:03:48 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:48 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.65@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:48.829945 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:48.820682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.65:49061)
0926 15:03:48.821213 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:48.821229 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:48.821385 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:48.821715 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:48.821718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:48.822109 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:48.822290 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:48.822299 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:48.823575 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:48.823581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:48.824690 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:48.824706 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:48.825565 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:48.825570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:48.825749 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:48.826372 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:48.826391 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:48.829677 (+ 3286us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":276,"threads_started":1}
W20260926 15:03:48.830298 19029 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:48.820842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:49061 (local address 127.9.123.126:39341)
0926 15:03:48.821207 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:48.821211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:48.821312 (+ 101us) server_negotiation.cc:407] Connection header received
0926 15:03:48.821505 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:48.821509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:48.821583 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:48.821676 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:48.822461 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:48.823396 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:48.824890 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:48.825394 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:48.825501 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:48.830018 (+ 4517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.65:49061: BlockingRecv error: recv got EOF from 127.9.123.65:49061 (error 108)
Metrics: {"server-negotiator.queue_time_us":190}
Sep 26 15:03:49 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
Sep 26 15:03:49 dist-test-slave-7153 krb5kdc[17970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.123.66@KRBTEST.COM for oryx/127.9.123.126@KRBTEST.COM, Server not found in Kerberos database
I20260926 15:03:49.434958 19032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:49.426362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39341 (local address 127.9.123.66:40937)
0926 15:03:49.426898 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:49.426915 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:49.427037 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:49.427380 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:49.427384 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:49.427747 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:49.427941 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:49.427951 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:49.428898 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:49.428902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:49.429866 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:49.429878 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:49.430550 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:49.430555 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:49.430733 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:49.431397 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:49.431416 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:49.434684 (+ 3268us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.123.126:39341: Server oryx/127.9.123.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":229,"threads_started":1}
W20260926 15:03:49.435084 19033 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:49.426487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:40937 (local address 127.9.123.126:39341)
0926 15:03:49.426934 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:49.426938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:49.426972 (+ 34us) server_negotiation.cc:407] Connection header received
0926 15:03:49.427128 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:49.427132 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:49.427189 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:49.427276 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:49.428060 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:49.428777 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:49.429989 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:49.430404 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:49.430495 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:49.434890 (+ 4395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.123.66:40937: BlockingRecv error: recv got EOF from 127.9.123.66:40937 (error 108)
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":238,"threads_started":1}
I20260926 15:03:49.518541 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 18558
I20260926 15:03:49.535552 19035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:49.520061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40552 (local address 127.9.123.65:41845)
0926 15:03:49.520725 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:49.520732 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:49.520783 (+ 51us) server_negotiation.cc:407] Connection header received
0926 15:03:49.520849 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:49.520856 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:49.521175 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:49.521420 (+ 245us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:49.523309 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:49.525378 (+ 2069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:49.527338 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:49.528138 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:49.528305 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:49.531950 (+ 3645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:49.531966 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:49.531986 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:49.532039 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:49.534988 (+ 2949us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.123.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0926 15:03:49.535335 (+ 347us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40552: Request ticket server kudu/127.9.123.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":366,"threads_started":1}
W20260926 15:03:49.535650 19035 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40552: Request ticket server kudu/127.9.123.65@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260926 15:03:49.535897 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 18558: Not authorized: Client connection negotiation failed: client connection to 127.9.123.65:41845: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.123.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260926 15:03:49.536135 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 18558
I20260926 15:03:49.552291 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 18710
I20260926 15:03:49.567085 19036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:49.553582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60542 (local address 127.9.123.66:42873)
0926 15:03:49.554308 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:49.554317 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:49.554357 (+ 40us) server_negotiation.cc:407] Connection header received
0926 15:03:49.554438 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:49.554447 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:49.554800 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:49.555062 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:49.556421 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:49.557772 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:49.559137 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:49.559791 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:49.559948 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:49.562865 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:49.562875 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:49.562889 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:49.562935 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:49.566476 (+ 3541us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.123.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0926 15:03:49.566837 (+ 361us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60542: Request ticket server kudu/127.9.123.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":351,"threads_started":1}
W20260926 15:03:49.567178 19036 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60542: Request ticket server kudu/127.9.123.66@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260926 15:03:49.567487 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 18710: Not authorized: Client connection negotiation failed: client connection to 127.9.123.66:42873: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.123.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260926 15:03:49.567617 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 18710
I20260926 15:03:49.582880 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 18843
I20260926 15:03:49.595527 19037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:49.583962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38600 (local address 127.9.123.67:35349)
0926 15:03:49.584543 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:49.584551 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:49.584577 (+ 26us) server_negotiation.cc:407] Connection header received
0926 15:03:49.584639 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:49.584646 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:49.584902 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:49.585092 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:49.586521 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:49.587588 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:49.588764 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:49.589401 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:49.589573 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:49.592192 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:49.592202 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:49.592217 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:49.592265 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:49.594809 (+ 2544us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.123.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0926 15:03:49.595183 (+ 374us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38600: Request ticket server kudu/127.9.123.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":286,"threads_started":1}
W20260926 15:03:49.595619 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 18843: Not authorized: Client connection negotiation failed: client connection to 127.9.123.67:35349: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.123.67@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260926 15:03:49.595650 19037 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38600: Request ticket server kudu/127.9.123.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260926 15:03:49.595784 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 18843
I20260926 15:03:49.608914 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 18489
I20260926 15:03:49.610517 18551 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:49.722957 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 18489
2026-09-26T15:03:49Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21021 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-26T15:03:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:03:49Z Disabled control of system clock
I20260926 15:03:49.791987 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:37483
--webserver_interface=127.9.123.126
--webserver_port=0
--builtin_ntp_servers=127.9.123.84:38421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:37483
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260926 15:03:50.111598 19042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:50.111903 19042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:50.112001 19042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:50.119196 19042 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:03:50.119325 19042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:50.119361 19042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:50.119424 19042 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:03:50.119486 19042 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:03:50.128680 19042 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:38421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:37483
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:37483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:50.130013 19042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:50.131837 19042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:50.143482 19047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:50.143591 19050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:50.143612 19048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:50.144315 19042 server_base.cc:1034] running on GCE node
I20260926 15:03:50.145210 19042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:50.146827 19042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:50.148129 19042 hybrid_clock.cc:648] HybridClock initialized: now 1790435030148075 us; error 37 us; skew 500 ppm
I20260926 15:03:50.148501 19042 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:03:50.150844 19042 webserver.cc:460] Webserver started at http://127.9.123.126:33565/ using document root <none> and password file <none>
I20260926 15:03:50.151571 19042 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:50.151672 19042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:50.151983 19042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:50.154942 19042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "66c50090063649b58828237a4a2ed9b5"
format_stamp: "Formatted at 2026-09-26 15:03:50 on dist-test-slave-7153"
server_key: "566a344778926559a01249e53b54bf5e"
server_key_iv: "dcf839257c06e15d56bad34b3bd05c95"
server_key_version: "encryptionkey@0"
I20260926 15:03:50.155720 19042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "66c50090063649b58828237a4a2ed9b5"
format_stamp: "Formatted at 2026-09-26 15:03:50 on dist-test-slave-7153"
server_key: "566a344778926559a01249e53b54bf5e"
server_key_iv: "dcf839257c06e15d56bad34b3bd05c95"
server_key_version: "encryptionkey@0"
I20260926 15:03:50.160843 19042 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 15:03:50.164549 19056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:50.166010 19042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:03:50.166181 19042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "66c50090063649b58828237a4a2ed9b5"
format_stamp: "Formatted at 2026-09-26 15:03:50 on dist-test-slave-7153"
server_key: "566a344778926559a01249e53b54bf5e"
server_key_iv: "dcf839257c06e15d56bad34b3bd05c95"
server_key_version: "encryptionkey@0"
I20260926 15:03:50.166388 19042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:50.189456 19042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:50.190333 19042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:50.190610 19042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:50.207715 19042 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:37483
I20260926 15:03:50.207736 19107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:37483 every 8 connection(s)
I20260926 15:03:50.209012 19042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:03:50.212817 19108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:50.212883 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 19042
I20260926 15:03:50.213059 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:03:50.213701 9709 external_mini_cluster.cc:1428] Setting key 7c401e6d52b84f738a3863cf117e9574
I20260926 15:03:50.227212 19108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66c50090063649b58828237a4a2ed9b5: Bootstrap starting.
I20260926 15:03:50.230063 19111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:50.215907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43842 (local address 127.9.123.126:37483)
0926 15:03:50.217030 (+ 1123us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:50.217045 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:50.217129 (+ 84us) server_negotiation.cc:407] Connection header received
0926 15:03:50.219321 (+ 2192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:50.219355 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:50.220106 (+ 751us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:50.220630 (+ 524us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:50.221205 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:50.223063 (+ 1858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:50.224501 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:50.225621 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:50.225807 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:50.226996 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:50.227006 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:50.227028 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:50.227780 (+ 752us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:50.227890 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:50.227915 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:50.228015 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:50.228269 (+ 254us) server_negotiation.cc:299] Negotiation successful
0926 15:03:50.228624 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":316,"threads_started":1}
I20260926 15:03:50.231346 19108 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66c50090063649b58828237a4a2ed9b5: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:50.232441 19108 log.cc:824] T 00000000000000000000000000000000 P 66c50090063649b58828237a4a2ed9b5: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:50.235442 19108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66c50090063649b58828237a4a2ed9b5: No bootstrap required, opened a new log
I20260926 15:03:50.242734 19108 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66c50090063649b58828237a4a2ed9b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c50090063649b58828237a4a2ed9b5" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 37483 } }
I20260926 15:03:50.243150 19108 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66c50090063649b58828237a4a2ed9b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:50.243250 19108 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66c50090063649b58828237a4a2ed9b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66c50090063649b58828237a4a2ed9b5, State: Initialized, Role: FOLLOWER
I20260926 15:03:50.243849 19108 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66c50090063649b58828237a4a2ed9b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c50090063649b58828237a4a2ed9b5" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 37483 } }
I20260926 15:03:50.244073 19108 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66c50090063649b58828237a4a2ed9b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:03:50.244195 19108 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66c50090063649b58828237a4a2ed9b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:03:50.244343 19108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66c50090063649b58828237a4a2ed9b5 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:50.246788 19108 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66c50090063649b58828237a4a2ed9b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c50090063649b58828237a4a2ed9b5" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 37483 } }
I20260926 15:03:50.247310 19108 leader_election.cc:302] T 00000000000000000000000000000000 P 66c50090063649b58828237a4a2ed9b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66c50090063649b58828237a4a2ed9b5; no voters:
I20260926 15:03:50.247957 19108 leader_election.cc:288] T 00000000000000000000000000000000 P 66c50090063649b58828237a4a2ed9b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:03:50.248100 19113 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66c50090063649b58828237a4a2ed9b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:03:50.248895 19113 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66c50090063649b58828237a4a2ed9b5 [term 1 LEADER]: Becoming Leader. State: Replica: 66c50090063649b58828237a4a2ed9b5, State: Running, Role: LEADER
I20260926 15:03:50.249559 19113 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66c50090063649b58828237a4a2ed9b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c50090063649b58828237a4a2ed9b5" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 37483 } }
I20260926 15:03:50.250099 19108 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66c50090063649b58828237a4a2ed9b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:03:50.252213 19115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66c50090063649b58828237a4a2ed9b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66c50090063649b58828237a4a2ed9b5. Latest consensus state: current_term: 1 leader_uuid: "66c50090063649b58828237a4a2ed9b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c50090063649b58828237a4a2ed9b5" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 37483 } } }
I20260926 15:03:50.252496 19115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66c50090063649b58828237a4a2ed9b5 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:50.252702 19114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66c50090063649b58828237a4a2ed9b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66c50090063649b58828237a4a2ed9b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66c50090063649b58828237a4a2ed9b5" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 37483 } } }
I20260926 15:03:50.253394 19114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66c50090063649b58828237a4a2ed9b5 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:50.255134 19121 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:03:50.261024 19121 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:03:50.270262 19121 catalog_manager.cc:1347] Generated new cluster ID: 5a54ed571252482fad361cf57d8fa608
I20260926 15:03:50.270401 19121 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:03:50.278255 19121 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:03:50.279557 19121 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:03:50.298941 19121 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66c50090063649b58828237a4a2ed9b5: Generated new TSK 0
I20260926 15:03:50.299829 19121 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:03:50.305716 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:37483
--builtin_ntp_servers=127.9.123.84:38421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260926 15:03:50.631697 19132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:50.631982 19132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:50.632090 19132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:50.639362 19132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:50.639470 19132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:50.639600 19132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:03:50.648765 19132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:38421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:37483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:50.650130 19132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:50.651988 19132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:50.665071 19140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:50.665179 19137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:50.665187 19138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:50.665730 19132 server_base.cc:1034] running on GCE node
I20260926 15:03:50.666445 19132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:50.667450 19132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:50.668666 19132 hybrid_clock.cc:648] HybridClock initialized: now 1790435030668618 us; error 38 us; skew 500 ppm
I20260926 15:03:50.669006 19132 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:03:50.671432 19132 webserver.cc:460] Webserver started at http://127.9.123.65:40791/ using document root <none> and password file <none>
I20260926 15:03:50.672194 19132 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:50.672287 19132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:50.672643 19132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:50.675485 19132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "7df56b9bfa8d493b9cb2b52e69fb1c6b"
format_stamp: "Formatted at 2026-09-26 15:03:50 on dist-test-slave-7153"
server_key: "fc1dc9cbe28df3ad482de2d963a2f057"
server_key_iv: "bb804d76d4970fe164b29eb536e865b8"
server_key_version: "encryptionkey@0"
I20260926 15:03:50.676225 19132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "7df56b9bfa8d493b9cb2b52e69fb1c6b"
format_stamp: "Formatted at 2026-09-26 15:03:50 on dist-test-slave-7153"
server_key: "fc1dc9cbe28df3ad482de2d963a2f057"
server_key_iv: "bb804d76d4970fe164b29eb536e865b8"
server_key_version: "encryptionkey@0"
I20260926 15:03:50.681543 19132 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 15:03:50.684923 19146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:50.686405 19132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:03:50.686579 19132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "7df56b9bfa8d493b9cb2b52e69fb1c6b"
format_stamp: "Formatted at 2026-09-26 15:03:50 on dist-test-slave-7153"
server_key: "fc1dc9cbe28df3ad482de2d963a2f057"
server_key_iv: "bb804d76d4970fe164b29eb536e865b8"
server_key_version: "encryptionkey@0"
I20260926 15:03:50.686784 19132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:50.728434 19132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:50.729357 19132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:50.729655 19132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:50.730644 19132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:50.732142 19132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:50.732230 19132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:50.732334 19132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:50.732419 19132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:50.762315 19132 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:45213
I20260926 15:03:50.762391 19258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:45213 every 8 connection(s)
I20260926 15:03:50.763659 19132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:03:50.769137 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 19132
I20260926 15:03:50.769315 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:03:50.769989 9709 external_mini_cluster.cc:1428] Setting key d637e3e1c8a7d9876207c8f34988da7d
I20260926 15:03:50.772962 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:37483
--builtin_ntp_servers=127.9.123.84:38421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 15:03:50.783340 19261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:50.766215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:33501 (local address 127.9.123.126:37483)
0926 15:03:50.766622 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:50.766627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:50.767870 (+ 1243us) server_negotiation.cc:407] Connection header received
0926 15:03:50.769645 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:50.769650 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:50.769733 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:50.769826 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:50.771105 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:50.772104 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:50.774484 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:50.774884 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:50.774991 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:50.782460 (+ 7469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:50.782466 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:50.782469 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:50.782490 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:50.782532 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:50.782545 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:50.782622 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:50.782981 (+ 359us) server_negotiation.cc:299] Negotiation successful
0926 15:03:50.783078 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":207,"threads_started":1}
I20260926 15:03:50.784718 19262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:50.766833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:37483 (local address 127.9.123.65:33501)
0926 15:03:50.767720 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:50.767765 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:50.769365 (+ 1600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:50.770092 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:50.770105 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:50.770224 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:50.770924 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:50.770944 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:50.772324 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:50.772329 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:50.774316 (+ 1987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:50.774333 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:50.775102 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:50.775108 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:50.775387 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:50.781973 (+ 6586us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 15:03:50.781978 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:50.782043 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 15:03:50.782175 (+ 132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:50.782650 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:50.782662 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:50.782670 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 15:03:50.782874 (+ 204us) client_negotiation.cc:240] Negotiation successful
0926 15:03:50.783345 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":244,"threads_started":1}
I20260926 15:03:50.786329 19259 heartbeater.cc:347] Connected to a master server at 127.9.123.126:37483
I20260926 15:03:50.786657 19259 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:50.787482 19259 heartbeater.cc:511] Master 127.9.123.126:37483 requested a full tablet report, sending...
I20260926 15:03:50.789639 19073 ts_manager.cc:194] Registered new tserver with Master: 7df56b9bfa8d493b9cb2b52e69fb1c6b (127.9.123.65:45213)
I20260926 15:03:50.791260 19073 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.123.65:33501
W20260926 15:03:51.100935 19263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:51.101307 19263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:51.101444 19263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:51.108891 19263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:51.109030 19263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:51.109217 19263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:03:51.118721 19263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:38421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:37483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:51.120054 19263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:51.121935 19263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:51.135608 19269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:51.135608 19271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:51.135697 19268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:51.136394 19263 server_base.cc:1034] running on GCE node
I20260926 15:03:51.137043 19263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:51.138144 19263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:51.139348 19263 hybrid_clock.cc:648] HybridClock initialized: now 1790435031139293 us; error 71 us; skew 500 ppm
I20260926 15:03:51.139737 19263 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:03:51.142227 19263 webserver.cc:460] Webserver started at http://127.9.123.66:42247/ using document root <none> and password file <none>
I20260926 15:03:51.142972 19263 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:51.143066 19263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:51.143343 19263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:51.146350 19263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "4618164e94ee43fbb548bc58cef2d6e7"
format_stamp: "Formatted at 2026-09-26 15:03:51 on dist-test-slave-7153"
server_key: "51f229dd76585411281ce05065ecaade"
server_key_iv: "765ef38aeeb7c9f8f09906e7d2a579d8"
server_key_version: "encryptionkey@0"
I20260926 15:03:51.147156 19263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "4618164e94ee43fbb548bc58cef2d6e7"
format_stamp: "Formatted at 2026-09-26 15:03:51 on dist-test-slave-7153"
server_key: "51f229dd76585411281ce05065ecaade"
server_key_iv: "765ef38aeeb7c9f8f09906e7d2a579d8"
server_key_version: "encryptionkey@0"
I20260926 15:03:51.152686 19263 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260926 15:03:51.156477 19277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:51.158102 19263 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:03:51.158280 19263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "4618164e94ee43fbb548bc58cef2d6e7"
format_stamp: "Formatted at 2026-09-26 15:03:51 on dist-test-slave-7153"
server_key: "51f229dd76585411281ce05065ecaade"
server_key_iv: "765ef38aeeb7c9f8f09906e7d2a579d8"
server_key_version: "encryptionkey@0"
I20260926 15:03:51.158483 19263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:51.175025 19263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:51.175930 19263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:51.176184 19263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:51.177126 19263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:51.178594 19263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:51.178681 19263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:51.178763 19263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:51.178815 19263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:51.209798 19263 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:41405
I20260926 15:03:51.209815 19389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:41405 every 8 connection(s)
I20260926 15:03:51.211422 19263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:03:51.216603 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 19263
I20260926 15:03:51.216780 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:03:51.217384 9709 external_mini_cluster.cc:1428] Setting key 7bd803f75c727e3b0236ca7a4fc680f4
I20260926 15:03:51.220671 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:37483
--builtin_ntp_servers=127.9.123.84:38421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260926 15:03:51.226686 19261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:51.214169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:37849 (local address 127.9.123.126:37483)
0926 15:03:51.214328 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:51.214332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:51.216136 (+ 1804us) server_negotiation.cc:407] Connection header received
0926 15:03:51.218592 (+ 2456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:51.218598 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:51.218684 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:51.218774 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:51.220252 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:51.221317 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:51.223619 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:51.224272 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:51.224399 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:51.226013 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:51.226018 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:51.226022 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:51.226042 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:51.226074 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:51.226086 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:51.226158 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:51.226380 (+ 222us) server_negotiation.cc:299] Negotiation successful
0926 15:03:51.226465 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 15:03:51.228266 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:51.214882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:37483 (local address 127.9.123.66:37849)
0926 15:03:51.215938 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:51.215985 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:51.218205 (+ 2220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:51.219054 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:51.219064 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:51.219176 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:51.220039 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:51.220066 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:51.221543 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:51.221549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:51.223457 (+ 1908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:51.223471 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:51.224527 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:51.224533 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:51.224846 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:51.225626 (+ 780us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 15:03:51.225630 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:51.225689 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 15:03:51.225802 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:51.226174 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:51.226185 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:51.226192 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 15:03:51.226385 (+ 193us) client_negotiation.cc:240] Negotiation successful
0926 15:03:51.226833 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":317,"threads_started":1}
I20260926 15:03:51.229947 19390 heartbeater.cc:347] Connected to a master server at 127.9.123.126:37483
I20260926 15:03:51.230319 19390 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:51.231400 19390 heartbeater.cc:511] Master 127.9.123.126:37483 requested a full tablet report, sending...
I20260926 15:03:51.233292 19073 ts_manager.cc:194] Registered new tserver with Master: 4618164e94ee43fbb548bc58cef2d6e7 (127.9.123.66:41405)
I20260926 15:03:51.234347 19073 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.123.66:37849
W20260926 15:03:51.560354 19393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:51.560650 19393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:51.560736 19393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:51.568483 19393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:51.568619 19393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:51.568769 19393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:03:51.578298 19393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:38421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:37483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:51.579562 19393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:51.581410 19393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:51.594708 19401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:51.594775 19398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:51.594749 19399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:51.595558 19393 server_base.cc:1034] running on GCE node
I20260926 15:03:51.596199 19393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:51.597268 19393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:51.598487 19393 hybrid_clock.cc:648] HybridClock initialized: now 1790435031598422 us; error 55 us; skew 500 ppm
I20260926 15:03:51.598822 19393 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:03:51.601240 19393 webserver.cc:460] Webserver started at http://127.9.123.67:39569/ using document root <none> and password file <none>
I20260926 15:03:51.601990 19393 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:51.602084 19393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:51.602406 19393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:51.605507 19393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "a70b5c7319d9408f9e574787d43d4f6a"
format_stamp: "Formatted at 2026-09-26 15:03:51 on dist-test-slave-7153"
server_key: "6d4fc5168400b495cf3959babc91d732"
server_key_iv: "835ac95431c095cefa8d11e7b54dae51"
server_key_version: "encryptionkey@0"
I20260926 15:03:51.606386 19393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "a70b5c7319d9408f9e574787d43d4f6a"
format_stamp: "Formatted at 2026-09-26 15:03:51 on dist-test-slave-7153"
server_key: "6d4fc5168400b495cf3959babc91d732"
server_key_iv: "835ac95431c095cefa8d11e7b54dae51"
server_key_version: "encryptionkey@0"
I20260926 15:03:51.612088 19393 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 15:03:51.616004 19407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:51.617628 19393 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:03:51.617811 19393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "a70b5c7319d9408f9e574787d43d4f6a"
format_stamp: "Formatted at 2026-09-26 15:03:51 on dist-test-slave-7153"
server_key: "6d4fc5168400b495cf3959babc91d732"
server_key_iv: "835ac95431c095cefa8d11e7b54dae51"
server_key_version: "encryptionkey@0"
I20260926 15:03:51.618036 19393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:51.631548 19393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:51.632490 19393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:51.632782 19393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:51.633796 19393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:51.635324 19393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:51.635416 19393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:51.635522 19393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:51.635586 19393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:51.665654 19393 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:38959
I20260926 15:03:51.665674 19519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:38959 every 8 connection(s)
I20260926 15:03:51.667047 19393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:03:51.668016 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 19393
I20260926 15:03:51.668172 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:03:51.668613 9709 external_mini_cluster.cc:1428] Setting key 4765ef3cae2a9ebfe513739096bbfd18
I20260926 15:03:51.679405 19261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:51.669619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:45225 (local address 127.9.123.126:37483)
0926 15:03:51.669804 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:51.669810 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:51.671524 (+ 1714us) server_negotiation.cc:407] Connection header received
0926 15:03:51.673236 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:51.673242 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:51.673320 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:51.673402 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:51.674573 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:51.675282 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:51.676768 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:51.677215 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:51.677322 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:51.678608 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:51.678613 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:51.678616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:51.678636 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:51.678665 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:51.678676 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:51.678746 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:51.678961 (+ 215us) server_negotiation.cc:299] Negotiation successful
0926 15:03:51.679089 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 15:03:51.680398 19522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:51.670287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:37483 (local address 127.9.123.67:45225)
0926 15:03:51.671359 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:51.671419 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:51.672973 (+ 1554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:51.673645 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:51.673653 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:51.673752 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:51.674418 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:51.674437 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:51.675425 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:51.675429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:51.676638 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:51.676650 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:51.677442 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:51.677450 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:51.677659 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:51.678293 (+ 634us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 15:03:51.678295 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:51.678340 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 15:03:51.678429 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:51.678756 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:51.678767 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:51.678775 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 15:03:51.678942 (+ 167us) client_negotiation.cc:240] Negotiation successful
0926 15:03:51.679370 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":304,"threads_started":1}
I20260926 15:03:51.681761 19520 heartbeater.cc:347] Connected to a master server at 127.9.123.126:37483
I20260926 15:03:51.682044 19520 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:51.682942 19520 heartbeater.cc:511] Master 127.9.123.126:37483 requested a full tablet report, sending...
I20260926 15:03:51.684404 19073 ts_manager.cc:194] Registered new tserver with Master: a70b5c7319d9408f9e574787d43d4f6a (127.9.123.67:38959)
I20260926 15:03:51.685200 19073 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.123.67:45225
I20260926 15:03:51.696282 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 15:03:51.702301 19261 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:51.700775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43850 (local address 127.9.123.126:37483)
0926 15:03:51.701027 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:51.701033 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:51.701424 (+ 391us) server_negotiation.cc:407] Connection header received
0926 15:03:51.701595 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:51.701598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:51.701658 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:51.701736 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:51.702065 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43850: BlockingRecv error: recv got EOF from 127.0.0.1:43850 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 15:03:51.703178 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 19132
I20260926 15:03:51.710862 19529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:51.704036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56440 (local address 127.9.123.65:45213)
0926 15:03:51.704519 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:51.704526 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:51.704549 (+ 23us) server_negotiation.cc:407] Connection header received
0926 15:03:51.704600 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:51.704606 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:51.704812 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:51.704954 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:51.705354 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:51.706331 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:51.707626 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:51.708207 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:51.708321 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:51.709408 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:51.709417 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:51.709431 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:51.710030 (+ 599us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:51.710078 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:51.710098 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:51.710197 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:51.710392 (+ 195us) server_negotiation.cc:299] Negotiation successful
0926 15:03:51.710529 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":215,"threads_started":1}
I20260926 15:03:51.712226 19254 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:51.793203 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 19132
I20260926 15:03:51.806329 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 19263
I20260926 15:03:51.813939 19531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:51.807183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59466 (local address 127.9.123.66:41405)
0926 15:03:51.807684 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:51.807691 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:51.807737 (+ 46us) server_negotiation.cc:407] Connection header received
0926 15:03:51.807803 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:51.807810 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:51.808041 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:51.808172 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:51.808516 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:51.809550 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:51.810834 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:51.811412 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:51.811516 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:51.812513 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:51.812521 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:51.812536 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:51.813197 (+ 661us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:51.813259 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:51.813278 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:51.813376 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:51.813517 (+ 141us) server_negotiation.cc:299] Negotiation successful
0926 15:03:51.813634 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":228,"threads_started":1}
I20260926 15:03:51.815253 19385 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:51.896772 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 19263
I20260926 15:03:51.909883 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 19393
I20260926 15:03:51.918272 19533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:51.910950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46456 (local address 127.9.123.67:38959)
0926 15:03:51.911607 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:51.911615 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:51.911646 (+ 31us) server_negotiation.cc:407] Connection header received
0926 15:03:51.911705 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:51.911711 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:51.911945 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:51.912109 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:51.912570 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:51.913657 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:51.914960 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:51.915536 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:51.915653 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:51.916739 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:51.916749 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:51.916764 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:51.917437 (+ 673us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:51.917487 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:51.917507 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:51.917621 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:51.917776 (+ 155us) server_negotiation.cc:299] Negotiation successful
0926 15:03:51.917930 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":337,"threads_started":1}
I20260926 15:03:51.919600 19515 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:52.001083 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 19393
I20260926 15:03:52.013636 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 19042
I20260926 15:03:52.014847 19103 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:52.114207 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 19042
2026-09-26T15:03:52Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2387 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-26T15:03:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:03:52Z Disabled control of system clock
I20260926 15:03:52.180478 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:45935
--webserver_interface=127.9.123.126
--webserver_port=0
--builtin_ntp_servers=127.9.123.84:33663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:45935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 15:03:52.513787 19539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:52.514102 19539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:52.514211 19539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:52.521550 19539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:03:52.521665 19539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:52.521703 19539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:52.521780 19539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:03:52.521833 19539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:03:52.531586 19539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:33663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:45935
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:45935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:52.532969 19539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:52.534835 19539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:52.546794 19544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:52.546682 19545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:52.546715 19547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:52.547540 19539 server_base.cc:1034] running on GCE node
I20260926 15:03:52.548462 19539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:52.550215 19539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:52.551457 19539 hybrid_clock.cc:648] HybridClock initialized: now 1790435032551410 us; error 48 us; skew 500 ppm
I20260926 15:03:52.551847 19539 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:03:52.554457 19539 webserver.cc:460] Webserver started at http://127.9.123.126:39033/ using document root <none> and password file <none>
I20260926 15:03:52.555200 19539 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:52.555312 19539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:52.555639 19539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:52.558779 19539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "e0cc828d5e2a4d1fb1195426735c7439"
format_stamp: "Formatted at 2026-09-26 15:03:52 on dist-test-slave-7153"
server_key: "abd5a2978ba3f3e9d2ae1efdd72e9fa0"
server_key_iv: "fda0460bf9e66ab5da92af5b2bc4c06f"
server_key_version: "encryptionkey@0"
I20260926 15:03:52.559602 19539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "e0cc828d5e2a4d1fb1195426735c7439"
format_stamp: "Formatted at 2026-09-26 15:03:52 on dist-test-slave-7153"
server_key: "abd5a2978ba3f3e9d2ae1efdd72e9fa0"
server_key_iv: "fda0460bf9e66ab5da92af5b2bc4c06f"
server_key_version: "encryptionkey@0"
I20260926 15:03:52.565078 19539 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 15:03:52.568729 19553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:52.570281 19539 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260926 15:03:52.570477 19539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "e0cc828d5e2a4d1fb1195426735c7439"
format_stamp: "Formatted at 2026-09-26 15:03:52 on dist-test-slave-7153"
server_key: "abd5a2978ba3f3e9d2ae1efdd72e9fa0"
server_key_iv: "fda0460bf9e66ab5da92af5b2bc4c06f"
server_key_version: "encryptionkey@0"
I20260926 15:03:52.570703 19539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:52.576586 19539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:52.577474 19539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:52.577773 19539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:52.594588 19539 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:45935
I20260926 15:03:52.594606 19604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:45935 every 8 connection(s)
I20260926 15:03:52.595959 19539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:03:52.599731 19605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:52.601912 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 19539
I20260926 15:03:52.602142 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:03:52.602881 9709 external_mini_cluster.cc:1428] Setting key 81ff88bda189d9c3f88434d7fd04b58a
I20260926 15:03:52.612025 19608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:52.605236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57870 (local address 127.9.123.126:45935)
0926 15:03:52.606315 (+ 1079us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:52.606328 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:52.606375 (+ 47us) server_negotiation.cc:407] Connection header received
0926 15:03:52.608129 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:52.608158 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:52.608672 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:52.609163 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:52.610270 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:52.610277 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:52.610291 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:52.610308 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:52.610368 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:52.610385 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:52.610626 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:52.610767 (+ 141us) server_negotiation.cc:299] Negotiation successful
0926 15:03:52.611029 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":372,"threads_started":1}
I20260926 15:03:52.612377 19605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0cc828d5e2a4d1fb1195426735c7439: Bootstrap starting.
I20260926 15:03:52.616176 19605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0cc828d5e2a4d1fb1195426735c7439: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:52.617300 19605 log.cc:824] T 00000000000000000000000000000000 P e0cc828d5e2a4d1fb1195426735c7439: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:52.620265 19605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0cc828d5e2a4d1fb1195426735c7439: No bootstrap required, opened a new log
I20260926 15:03:52.628046 19605 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0cc828d5e2a4d1fb1195426735c7439 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0cc828d5e2a4d1fb1195426735c7439" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45935 } }
I20260926 15:03:52.628448 19605 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0cc828d5e2a4d1fb1195426735c7439 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:52.628539 19605 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0cc828d5e2a4d1fb1195426735c7439 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0cc828d5e2a4d1fb1195426735c7439, State: Initialized, Role: FOLLOWER
I20260926 15:03:52.629185 19605 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0cc828d5e2a4d1fb1195426735c7439 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0cc828d5e2a4d1fb1195426735c7439" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45935 } }
I20260926 15:03:52.629405 19605 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0cc828d5e2a4d1fb1195426735c7439 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:03:52.629534 19605 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0cc828d5e2a4d1fb1195426735c7439 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:03:52.629683 19605 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0cc828d5e2a4d1fb1195426735c7439 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:52.631814 19605 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0cc828d5e2a4d1fb1195426735c7439 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0cc828d5e2a4d1fb1195426735c7439" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45935 } }
I20260926 15:03:52.632323 19605 leader_election.cc:302] T 00000000000000000000000000000000 P e0cc828d5e2a4d1fb1195426735c7439 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0cc828d5e2a4d1fb1195426735c7439; no voters:
I20260926 15:03:52.632930 19605 leader_election.cc:288] T 00000000000000000000000000000000 P e0cc828d5e2a4d1fb1195426735c7439 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:03:52.633075 19610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0cc828d5e2a4d1fb1195426735c7439 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:03:52.633744 19610 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0cc828d5e2a4d1fb1195426735c7439 [term 1 LEADER]: Becoming Leader. State: Replica: e0cc828d5e2a4d1fb1195426735c7439, State: Running, Role: LEADER
I20260926 15:03:52.634305 19610 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0cc828d5e2a4d1fb1195426735c7439 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0cc828d5e2a4d1fb1195426735c7439" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45935 } }
I20260926 15:03:52.635052 19605 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0cc828d5e2a4d1fb1195426735c7439 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:03:52.637372 19611 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0cc828d5e2a4d1fb1195426735c7439 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0cc828d5e2a4d1fb1195426735c7439" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0cc828d5e2a4d1fb1195426735c7439" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45935 } } }
I20260926 15:03:52.637822 19611 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0cc828d5e2a4d1fb1195426735c7439 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:52.637394 19612 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0cc828d5e2a4d1fb1195426735c7439 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0cc828d5e2a4d1fb1195426735c7439. Latest consensus state: current_term: 1 leader_uuid: "e0cc828d5e2a4d1fb1195426735c7439" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0cc828d5e2a4d1fb1195426735c7439" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45935 } } }
I20260926 15:03:52.637948 19612 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0cc828d5e2a4d1fb1195426735c7439 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:52.642510 19617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:03:52.648175 19617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:03:52.657598 19617 catalog_manager.cc:1347] Generated new cluster ID: 1330c4ef00204723895ca04121b3648a
I20260926 15:03:52.657720 19617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:03:52.677412 19617 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:03:52.677865 19617 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:03:52.688947 19617 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0cc828d5e2a4d1fb1195426735c7439: Generated new TSK 0
I20260926 15:03:52.690054 19617 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:03:52.700546 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:45935
--builtin_ntp_servers=127.9.123.84:33663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260926 15:03:53.039345 19629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:53.039666 19629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:53.039780 19629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:53.047286 19629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:53.047415 19629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:53.047575 19629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:03:53.056954 19629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:33663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:45935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:53.058384 19629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:53.060264 19629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:53.075918 19637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:53.076011 19635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:53.076030 19634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:53.077399 19629 server_base.cc:1034] running on GCE node
I20260926 15:03:53.078133 19629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:53.079223 19629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:53.080472 19629 hybrid_clock.cc:648] HybridClock initialized: now 1790435033080405 us; error 58 us; skew 500 ppm
I20260926 15:03:53.080946 19629 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:03:53.083959 19629 webserver.cc:460] Webserver started at http://127.9.123.65:44429/ using document root <none> and password file <none>
I20260926 15:03:53.084759 19629 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:53.084877 19629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:53.085251 19629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:53.091164 19629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "a5d467e826db4f94b04ac404c0ed0c1f"
format_stamp: "Formatted at 2026-09-26 15:03:53 on dist-test-slave-7153"
server_key: "10e09dd84bbf9c7268d7b4b3f3720828"
server_key_iv: "0bbb8072311e82d77758db753455ddae"
server_key_version: "encryptionkey@0"
I20260926 15:03:53.092173 19629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "a5d467e826db4f94b04ac404c0ed0c1f"
format_stamp: "Formatted at 2026-09-26 15:03:53 on dist-test-slave-7153"
server_key: "10e09dd84bbf9c7268d7b4b3f3720828"
server_key_iv: "0bbb8072311e82d77758db753455ddae"
server_key_version: "encryptionkey@0"
I20260926 15:03:53.098623 19629 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260926 15:03:53.102684 19643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:53.104496 19629 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260926 15:03:53.104702 19629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "a5d467e826db4f94b04ac404c0ed0c1f"
format_stamp: "Formatted at 2026-09-26 15:03:53 on dist-test-slave-7153"
server_key: "10e09dd84bbf9c7268d7b4b3f3720828"
server_key_iv: "0bbb8072311e82d77758db753455ddae"
server_key_version: "encryptionkey@0"
I20260926 15:03:53.104955 19629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:53.112049 19629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:53.113204 19629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:53.113523 19629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:53.114591 19629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:53.116219 19629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:53.116328 19629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:53.116451 19629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:53.116520 19629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:53.147604 19629 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:36851
I20260926 15:03:53.147651 19755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:36851 every 8 connection(s)
I20260926 15:03:53.148948 19629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:03:53.154371 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 19629
I20260926 15:03:53.154577 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:03:53.155200 9709 external_mini_cluster.cc:1428] Setting key 3acab7f26195b65842fd9e99d9582202
I20260926 15:03:53.157780 19758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:53.151577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:41293 (local address 127.9.123.126:45935)
0926 15:03:53.152027 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:53.152032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:53.153487 (+ 1455us) server_negotiation.cc:407] Connection header received
0926 15:03:53.155297 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:53.155302 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:53.155372 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:53.155475 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:53.157036 (+ 1561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:53.157040 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:53.157044 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:53.157059 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:53.157127 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:53.157142 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:53.157259 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:53.157444 (+ 185us) server_negotiation.cc:299] Negotiation successful
0926 15:03:53.157503 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":226,"threads_started":1}
I20260926 15:03:53.158232 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:45935
--builtin_ntp_servers=127.9.123.84:33663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260926 15:03:53.158773 19759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:53.152220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45935 (local address 127.9.123.65:41293)
0926 15:03:53.153309 (+ 1089us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:53.153355 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:53.154994 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:53.155784 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:53.155796 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:53.155958 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:53.156654 (+ 696us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 15:03:53.156657 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:53.156713 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 15:03:53.156842 (+ 129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:53.157253 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:53.157262 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:53.157268 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 15:03:53.157491 (+ 223us) client_negotiation.cc:240] Negotiation successful
0926 15:03:53.157722 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":284,"threads_started":1}
I20260926 15:03:53.160207 19756 heartbeater.cc:347] Connected to a master server at 127.9.123.126:45935
I20260926 15:03:53.160533 19756 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:53.161226 19756 heartbeater.cc:511] Master 127.9.123.126:45935 requested a full tablet report, sending...
I20260926 15:03:53.163676 19570 ts_manager.cc:194] Registered new tserver with Master: a5d467e826db4f94b04ac404c0ed0c1f (127.9.123.65:36851)
W20260926 15:03:53.496277 19760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:53.496577 19760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:53.496665 19760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:53.504127 19760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:53.504251 19760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:53.504360 19760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:03:53.513851 19760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:33663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:45935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:53.515110 19760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:53.516866 19760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:53.531088 19765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:53.531136 19768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:53.531136 19766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:53.533743 19760 server_base.cc:1034] running on GCE node
I20260926 15:03:53.534431 19760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:53.535475 19760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:53.536681 19760 hybrid_clock.cc:648] HybridClock initialized: now 1790435033536615 us; error 58 us; skew 500 ppm
I20260926 15:03:53.537005 19760 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:03:53.539543 19760 webserver.cc:460] Webserver started at http://127.9.123.66:42245/ using document root <none> and password file <none>
I20260926 15:03:53.540261 19760 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:53.540359 19760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:53.540679 19760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:53.543861 19760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "90868078b27b4a9da5c377a711f4e9ee"
format_stamp: "Formatted at 2026-09-26 15:03:53 on dist-test-slave-7153"
server_key: "3870ad591a08dc50f189849f0d4f0ea4"
server_key_iv: "c4db6bfaeef583af46fc0e69a1eb19a8"
server_key_version: "encryptionkey@0"
I20260926 15:03:53.544644 19760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "90868078b27b4a9da5c377a711f4e9ee"
format_stamp: "Formatted at 2026-09-26 15:03:53 on dist-test-slave-7153"
server_key: "3870ad591a08dc50f189849f0d4f0ea4"
server_key_iv: "c4db6bfaeef583af46fc0e69a1eb19a8"
server_key_version: "encryptionkey@0"
I20260926 15:03:53.550199 19760 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 15:03:53.553825 19774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:53.555275 19760 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:03:53.555460 19760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "90868078b27b4a9da5c377a711f4e9ee"
format_stamp: "Formatted at 2026-09-26 15:03:53 on dist-test-slave-7153"
server_key: "3870ad591a08dc50f189849f0d4f0ea4"
server_key_iv: "c4db6bfaeef583af46fc0e69a1eb19a8"
server_key_version: "encryptionkey@0"
I20260926 15:03:53.555670 19760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:53.561736 19760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:53.562553 19760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:53.562800 19760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:53.563932 19760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:53.565465 19760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:53.565560 19760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:53.565666 19760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:53.565734 19760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:53.599323 19760 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:46377
I20260926 15:03:53.599402 19886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:46377 every 8 connection(s)
I20260926 15:03:53.600687 19760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:03:53.602118 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 19760
I20260926 15:03:53.602294 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:03:53.602792 9709 external_mini_cluster.cc:1428] Setting key 125a87733022f67adba3aeb52765248e
I20260926 15:03:53.605903 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:45935
--builtin_ntp_servers=127.9.123.84:33663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260926 15:03:53.612357 19758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:53.605153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:36831 (local address 127.9.123.126:45935)
0926 15:03:53.605334 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:53.605341 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:53.607222 (+ 1881us) server_negotiation.cc:407] Connection header received
0926 15:03:53.609875 (+ 2653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:53.609881 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:53.609974 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:53.610103 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:53.611685 (+ 1582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:53.611691 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:53.611696 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:53.611715 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:53.611753 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:53.611767 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:53.611889 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:53.612099 (+ 210us) server_negotiation.cc:299] Negotiation successful
0926 15:03:53.612157 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 15:03:53.613684 19889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:53.605890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45935 (local address 127.9.123.66:36831)
0926 15:03:53.607058 (+ 1168us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:53.607110 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:53.609518 (+ 2408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:53.610381 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:53.610394 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:53.610537 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:53.611310 (+ 773us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 15:03:53.611314 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:53.611367 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 15:03:53.611487 (+ 120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:53.611867 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:53.611875 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:53.611883 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 15:03:53.612171 (+ 288us) client_negotiation.cc:240] Negotiation successful
0926 15:03:53.612476 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":310,"threads_started":1}
I20260926 15:03:53.615177 19887 heartbeater.cc:347] Connected to a master server at 127.9.123.126:45935
I20260926 15:03:53.615479 19887 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:53.616079 19887 heartbeater.cc:511] Master 127.9.123.126:45935 requested a full tablet report, sending...
I20260926 15:03:53.617630 19570 ts_manager.cc:194] Registered new tserver with Master: 90868078b27b4a9da5c377a711f4e9ee (127.9.123.66:46377)
W20260926 15:03:53.947273 19890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:53.947577 19890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:53.947665 19890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:53.955042 19890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:53.955168 19890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:53.955279 19890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:03:53.964771 19890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:33663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:45935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:53.966136 19890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:53.967940 19890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:53.981627 19895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:53.981657 19896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:53.981570 19898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:53.983078 19890 server_base.cc:1034] running on GCE node
I20260926 15:03:53.983788 19890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:53.984807 19890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:53.986013 19890 hybrid_clock.cc:648] HybridClock initialized: now 1790435033985935 us; error 75 us; skew 500 ppm
I20260926 15:03:53.986340 19890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:03:53.988703 19890 webserver.cc:460] Webserver started at http://127.9.123.67:39295/ using document root <none> and password file <none>
I20260926 15:03:53.989534 19890 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:53.989631 19890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:53.989943 19890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:53.992933 19890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "abb616fdef2f4d57bc173fb0fb7b06bc"
format_stamp: "Formatted at 2026-09-26 15:03:53 on dist-test-slave-7153"
server_key: "f89ed68b9f44fccc10af7dff0a0372fe"
server_key_iv: "c253ecbb79b3050e978eb8cb880ff7ec"
server_key_version: "encryptionkey@0"
I20260926 15:03:53.993853 19890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "abb616fdef2f4d57bc173fb0fb7b06bc"
format_stamp: "Formatted at 2026-09-26 15:03:53 on dist-test-slave-7153"
server_key: "f89ed68b9f44fccc10af7dff0a0372fe"
server_key_iv: "c253ecbb79b3050e978eb8cb880ff7ec"
server_key_version: "encryptionkey@0"
I20260926 15:03:53.999301 19890 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 15:03:54.003165 19904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:54.004655 19890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 15:03:54.004838 19890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "abb616fdef2f4d57bc173fb0fb7b06bc"
format_stamp: "Formatted at 2026-09-26 15:03:53 on dist-test-slave-7153"
server_key: "f89ed68b9f44fccc10af7dff0a0372fe"
server_key_iv: "c253ecbb79b3050e978eb8cb880ff7ec"
server_key_version: "encryptionkey@0"
I20260926 15:03:54.005070 19890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:54.011365 19890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:54.012272 19890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:54.012557 19890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:54.013572 19890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:54.015131 19890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:54.015235 19890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:54.015350 19890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:54.015429 19890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:54.053416 19890 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:39919
I20260926 15:03:54.053440 20016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:39919 every 8 connection(s)
I20260926 15:03:54.054800 19890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:03:54.060741 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 19890
I20260926 15:03:54.060971 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:03:54.061442 9709 external_mini_cluster.cc:1428] Setting key d2b4fca1b56ed6e63a8557d5202958d4
I20260926 15:03:54.063741 19758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:54.057471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:54533 (local address 127.9.123.126:45935)
0926 15:03:54.057683 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:54.057689 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:54.059266 (+ 1577us) server_negotiation.cc:407] Connection header received
0926 15:03:54.061083 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:54.061089 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:54.061202 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:54.061294 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:54.062927 (+ 1633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:54.062933 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:54.062937 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:54.062959 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:54.063001 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:54.063015 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:54.063146 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:54.063325 (+ 179us) server_negotiation.cc:299] Negotiation successful
0926 15:03:54.063437 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 15:03:54.064899 20019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:54.058064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45935 (local address 127.9.123.67:54533)
0926 15:03:54.059099 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:54.059146 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:54.060766 (+ 1620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:54.061567 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:54.061580 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:54.061702 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:54.062414 (+ 712us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 15:03:54.062417 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:54.062478 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 15:03:54.062610 (+ 132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:54.063114 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:54.063122 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:54.063129 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 15:03:54.063405 (+ 276us) client_negotiation.cc:240] Negotiation successful
0926 15:03:54.063722 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":296,"threads_started":1}
I20260926 15:03:54.066557 20017 heartbeater.cc:347] Connected to a master server at 127.9.123.126:45935
I20260926 15:03:54.066912 20017 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:54.067574 20017 heartbeater.cc:511] Master 127.9.123.126:45935 requested a full tablet report, sending...
I20260926 15:03:54.069016 19570 ts_manager.cc:194] Registered new tserver with Master: abb616fdef2f4d57bc173fb0fb7b06bc (127.9.123.67:39919)
I20260926 15:03:54.077234 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260926 15:03:54.083667 19758 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:03:54.082096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57872 (local address 127.9.123.126:45935)
0926 15:03:54.082288 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:54.082293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:54.082809 (+ 516us) server_negotiation.cc:407] Connection header received
0926 15:03:54.082956 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:54.082962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:54.083020 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:54.083110 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:54.083462 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57872: BlockingRecv error: recv got EOF from 127.0.0.1:57872 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 15:03:54.084440 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 19629
I20260926 15:03:54.088110 20026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:54.085379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48886 (local address 127.9.123.65:36851)
0926 15:03:54.085867 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:54.085881 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:54.085906 (+ 25us) server_negotiation.cc:407] Connection header received
0926 15:03:54.085972 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:54.085979 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:54.086190 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:54.086326 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:54.087292 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:54.087299 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:54.087312 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:54.087331 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:54.087374 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:54.087389 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:54.087596 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:54.087717 (+ 121us) server_negotiation.cc:299] Negotiation successful
0926 15:03:54.087838 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":210,"threads_started":1}
I20260926 15:03:54.089444 19751 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:54.169301 19756 heartbeater.cc:503] Master 127.9.123.126:45935 was elected leader, sending a full tablet report...
I20260926 15:03:54.173280 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 19629
I20260926 15:03:54.186954 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 19760
I20260926 15:03:54.190241 20028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:54.187855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32930 (local address 127.9.123.66:46377)
0926 15:03:54.188348 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:54.188355 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:54.188382 (+ 27us) server_negotiation.cc:407] Connection header received
0926 15:03:54.188442 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:54.188449 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:54.188661 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:54.188786 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:54.189628 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:54.189634 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:54.189646 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:54.189658 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:54.189698 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:54.189711 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:54.189815 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:54.189889 (+ 74us) server_negotiation.cc:299] Negotiation successful
0926 15:03:54.189966 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":219,"threads_started":1}
I20260926 15:03:54.191484 19882 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:54.272959 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 19760
I20260926 15:03:54.286932 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 19890
I20260926 15:03:54.291180 20030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:54.287937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57384 (local address 127.9.123.67:39919)
0926 15:03:54.288607 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:54.288615 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:54.288649 (+ 34us) server_negotiation.cc:407] Connection header received
0926 15:03:54.288714 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:54.288720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:54.289075 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:54.289325 (+ 250us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:54.290447 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:54.290454 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:54.290468 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:03:54.290487 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:54.290527 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:03:54.290541 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:54.290684 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:54.290780 (+ 96us) server_negotiation.cc:299] Negotiation successful
0926 15:03:54.290881 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":354,"threads_started":1}
I20260926 15:03:54.292639 20012 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:54.375767 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 19890
I20260926 15:03:54.389004 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 19539
I20260926 15:03:54.390399 19600 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:54.491226 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 19539
2026-09-26T15:03:54Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2378 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:03:54 dist-test-slave-7153 krb5kdc[20035](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:03:54 dist-test-slave-7153 krb5kdc[20035](info): set up 2 sockets
Sep 26 15:03:54 dist-test-slave-7153 krb5kdc[20035](info): commencing operation
krb5kdc: starting...
W20260926 15:03:55.522325 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.002s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:03:55 dist-test-slave-7153 krb5kdc[20035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:03:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:03:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:03:55.757483 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:42715
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:42715
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:03:56.095782 20048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:56.096110 20048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:56.096235 20048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:56.103477 20048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:03:56.103605 20048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:56.103672 20048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:56.103744 20048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:03:56.103801 20048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:03:56.113754 20048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:42715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:42715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:56.115180 20048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:56.116503 20048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:56.128362 20053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:56.128474 20056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:56.128324 20054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:56.129006 20048 server_base.cc:1034] running on GCE node
I20260926 15:03:56.129930 20048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:56.131356 20048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:56.132593 20048 hybrid_clock.cc:648] HybridClock initialized: now 1790435036132538 us; error 45 us; skew 500 ppm
I20260926 15:03:56.132948 20048 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:56 dist-test-slave-7153 krb5kdc[20035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435036, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:56.142418 20048 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:03:56.144012 20048 webserver.cc:460] Webserver started at http://127.9.123.126:40215/ using document root <none> and password file <none>
I20260926 15:03:56.144723 20048 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:56.144845 20048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:56.145186 20048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:56.148267 20048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "8435e35ba83a41aba628b9304a4eaff0"
format_stamp: "Formatted at 2026-09-26 15:03:56 on dist-test-slave-7153"
server_key: "431d2d6ceb28e83ccbd84b284c4f7499"
server_key_iv: "938a66df174e359dca7f5835cd050b8a"
server_key_version: "encryptionkey@0"
I20260926 15:03:56.149048 20048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "8435e35ba83a41aba628b9304a4eaff0"
format_stamp: "Formatted at 2026-09-26 15:03:56 on dist-test-slave-7153"
server_key: "431d2d6ceb28e83ccbd84b284c4f7499"
server_key_iv: "938a66df174e359dca7f5835cd050b8a"
server_key_version: "encryptionkey@0"
I20260926 15:03:56.154537 20048 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 15:03:56.158224 20063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:56.159644 20048 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 15:03:56.159825 20048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "8435e35ba83a41aba628b9304a4eaff0"
format_stamp: "Formatted at 2026-09-26 15:03:56 on dist-test-slave-7153"
server_key: "431d2d6ceb28e83ccbd84b284c4f7499"
server_key_iv: "938a66df174e359dca7f5835cd050b8a"
server_key_version: "encryptionkey@0"
I20260926 15:03:56.160029 20048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:56.176105 20048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:56.180447 20048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:56.180721 20048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:56.198426 20048 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:42715
I20260926 15:03:56.198472 20114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:42715 every 8 connection(s)
I20260926 15:03:56.199898 20048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:03:56.203995 20115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:56.204994 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 20048
I20260926 15:03:56.205253 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:03:56.206041 9709 external_mini_cluster.cc:1428] Setting key 69370746c102c216e1f2610266655eb3
I20260926 15:03:56.218914 20115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8435e35ba83a41aba628b9304a4eaff0: Bootstrap starting.
Sep 26 15:03:56 dist-test-slave-7153 krb5kdc[20035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:03:56.222968 20115 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8435e35ba83a41aba628b9304a4eaff0: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:56.224098 20115 log.cc:824] T 00000000000000000000000000000000 P 8435e35ba83a41aba628b9304a4eaff0: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:56.227753 20115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8435e35ba83a41aba628b9304a4eaff0: No bootstrap required, opened a new log
I20260926 15:03:56.231037 20118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:56.208243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39412 (local address 127.9.123.126:42715)
0926 15:03:56.209058 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:56.209072 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:56.209144 (+ 72us) server_negotiation.cc:407] Connection header received
0926 15:03:56.210791 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:56.210819 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:56.211276 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:56.211644 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:56.213465 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:56.214839 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:56.216748 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:56.217632 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:56.217790 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:56.222660 (+ 4870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:56.222670 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:56.222688 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:56.222734 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:56.226290 (+ 3556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:56.227278 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:56.227286 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:56.227293 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:56.227474 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:56.227987 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:56.227992 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:56.227996 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:56.228584 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:56.228876 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:56.229339 (+ 463us) server_negotiation.cc:299] Negotiation successful
0926 15:03:56.229777 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":365,"threads_started":1}
I20260926 15:03:56.236150 20115 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8435e35ba83a41aba628b9304a4eaff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8435e35ba83a41aba628b9304a4eaff0" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42715 } }
I20260926 15:03:56.236543 20115 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8435e35ba83a41aba628b9304a4eaff0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:56.236647 20115 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8435e35ba83a41aba628b9304a4eaff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8435e35ba83a41aba628b9304a4eaff0, State: Initialized, Role: FOLLOWER
I20260926 15:03:56.237273 20115 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8435e35ba83a41aba628b9304a4eaff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8435e35ba83a41aba628b9304a4eaff0" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42715 } }
I20260926 15:03:56.237497 20115 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8435e35ba83a41aba628b9304a4eaff0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:03:56.237622 20115 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8435e35ba83a41aba628b9304a4eaff0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:03:56.237774 20115 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8435e35ba83a41aba628b9304a4eaff0 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:56.240021 20115 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8435e35ba83a41aba628b9304a4eaff0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8435e35ba83a41aba628b9304a4eaff0" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42715 } }
I20260926 15:03:56.240643 20115 leader_election.cc:302] T 00000000000000000000000000000000 P 8435e35ba83a41aba628b9304a4eaff0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8435e35ba83a41aba628b9304a4eaff0; no voters:
I20260926 15:03:56.241303 20115 leader_election.cc:288] T 00000000000000000000000000000000 P 8435e35ba83a41aba628b9304a4eaff0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:03:56.241442 20120 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8435e35ba83a41aba628b9304a4eaff0 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:03:56.242066 20120 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8435e35ba83a41aba628b9304a4eaff0 [term 1 LEADER]: Becoming Leader. State: Replica: 8435e35ba83a41aba628b9304a4eaff0, State: Running, Role: LEADER
I20260926 15:03:56.242683 20120 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8435e35ba83a41aba628b9304a4eaff0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8435e35ba83a41aba628b9304a4eaff0" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42715 } }
I20260926 15:03:56.243623 20115 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8435e35ba83a41aba628b9304a4eaff0 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:03:56.245576 20121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8435e35ba83a41aba628b9304a4eaff0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8435e35ba83a41aba628b9304a4eaff0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8435e35ba83a41aba628b9304a4eaff0" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42715 } } }
I20260926 15:03:56.245988 20121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8435e35ba83a41aba628b9304a4eaff0 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:56.246418 20122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8435e35ba83a41aba628b9304a4eaff0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8435e35ba83a41aba628b9304a4eaff0. Latest consensus state: current_term: 1 leader_uuid: "8435e35ba83a41aba628b9304a4eaff0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8435e35ba83a41aba628b9304a4eaff0" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42715 } } }
I20260926 15:03:56.246734 20122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8435e35ba83a41aba628b9304a4eaff0 [sys.catalog]: This master's current role is: LEADER
I20260926 15:03:56.247571 20129 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:03:56.252938 20129 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:03:56.262986 20129 catalog_manager.cc:1347] Generated new cluster ID: 6dd4ab524d71416fbf3eaae4ebcb115f
I20260926 15:03:56.263095 20129 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:03:56.284070 20129 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:03:56.285502 20129 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:03:56.294559 20129 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8435e35ba83a41aba628b9304a4eaff0: Generated new TSK 0
I20260926 15:03:56.295517 20129 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.123.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
I20260926 15:03:56.349970 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:42715
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:03:56.675607 20141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:56.675905 20141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:56.675997 20141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:56.683260 20141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:56.683391 20141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:56.683502 20141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:03:56.693380 20141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:42715
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:56.694633 20141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:56.695871 20141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:56.709156 20147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:56.709314 20146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:56.709131 20149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:56.710011 20141 server_base.cc:1034] running on GCE node
I20260926 15:03:56.710673 20141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:56.711680 20141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:56.712893 20141 hybrid_clock.cc:648] HybridClock initialized: now 1790435036712845 us; error 40 us; skew 500 ppm
I20260926 15:03:56.713244 20141 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:56 dist-test-slave-7153 krb5kdc[20035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435036, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:56.716403 20141 init.cc:377] Logged in from keytab as kudu/127.9.123.65@KRBTEST.COM (short username kudu)
I20260926 15:03:56.717963 20141 webserver.cc:460] Webserver started at http://127.9.123.65:45279/ using document root <none> and password file <none>
I20260926 15:03:56.718688 20141 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:56.718792 20141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:56.719046 20141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:56.722230 20141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "d229a50bfc594523a5ec08e4b1a0d209"
format_stamp: "Formatted at 2026-09-26 15:03:56 on dist-test-slave-7153"
server_key: "dea69feefa840dc07f13a68a02f6dd3b"
server_key_iv: "30e7afd62ce001b5034c33f27c140ced"
server_key_version: "encryptionkey@0"
I20260926 15:03:56.723060 20141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "d229a50bfc594523a5ec08e4b1a0d209"
format_stamp: "Formatted at 2026-09-26 15:03:56 on dist-test-slave-7153"
server_key: "dea69feefa840dc07f13a68a02f6dd3b"
server_key_iv: "30e7afd62ce001b5034c33f27c140ced"
server_key_version: "encryptionkey@0"
I20260926 15:03:56.728422 20141 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260926 15:03:56.732116 20156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:56.733650 20141 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:03:56.733865 20141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "d229a50bfc594523a5ec08e4b1a0d209"
format_stamp: "Formatted at 2026-09-26 15:03:56 on dist-test-slave-7153"
server_key: "dea69feefa840dc07f13a68a02f6dd3b"
server_key_iv: "30e7afd62ce001b5034c33f27c140ced"
server_key_version: "encryptionkey@0"
I20260926 15:03:56.734066 20141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:56.755148 20141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:56.759411 20141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:56.759683 20141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:56.760679 20141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:56.762212 20141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:56.762291 20141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:56.762390 20141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:56.762434 20141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:56.792155 20141 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:33299
I20260926 15:03:56.792200 20268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:33299 every 8 connection(s)
I20260926 15:03:56.793465 20141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:03:56.801851 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 20141
I20260926 15:03:56.802068 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:03:56.802673 9709 external_mini_cluster.cc:1428] Setting key f48cb5c4d0ae27ea55398ca028dcf711
Sep 26 15:03:56 dist-test-slave-7153 krb5kdc[20035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435036, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:03:56.814194 20271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:56.796041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:38213 (local address 127.9.123.126:42715)
0926 15:03:56.796430 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:56.796435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:56.797488 (+ 1053us) server_negotiation.cc:407] Connection header received
0926 15:03:56.799284 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:56.799289 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:56.799375 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:56.799473 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:56.801086 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:56.801886 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:56.803682 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:56.804119 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:56.804234 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:56.807902 (+ 3668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:56.807909 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:56.807913 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:56.807954 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:56.810864 (+ 2910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:56.811725 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:56.811730 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:56.811732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:56.811897 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:56.812342 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:56.812347 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:56.812349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:56.812643 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:56.812777 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:56.813586 (+ 809us) server_negotiation.cc:299] Negotiation successful
0926 15:03:56.813821 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":191,"threads_started":1}
I20260926 15:03:56.815246 20272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:56.796624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:42715 (local address 127.9.123.65:38213)
0926 15:03:56.797335 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:56.797376 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:56.799011 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:56.799726 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:56.799735 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:56.800291 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:56.800899 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:56.800919 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:56.802060 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:56.802064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:56.803545 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:56.803558 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:56.804366 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:56.804374 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:56.804647 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:56.805406 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:56.805436 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:56.807622 (+ 2186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:56.811052 (+ 3430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:56.811060 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:56.811076 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:56.811574 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:56.812018 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:56.812021 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:56.812024 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:56.812224 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:56.812773 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:56.812782 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:56.813191 (+ 409us) client_negotiation.cc:769] Sending connection context
0926 15:03:56.813478 (+ 287us) client_negotiation.cc:240] Negotiation successful
0926 15:03:56.814067 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":251,"threads_started":1}
I20260926 15:03:56.816874 20269 heartbeater.cc:347] Connected to a master server at 127.9.123.126:42715
I20260926 15:03:56.817227 20269 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:56.818006 20269 heartbeater.cc:511] Master 127.9.123.126:42715 requested a full tablet report, sending...
I20260926 15:03:56.820466 20080 ts_manager.cc:194] Registered new tserver with Master: d229a50bfc594523a5ec08e4b1a0d209 (127.9.123.65:33299)
I20260926 15:03:56.822623 20080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.65@KRBTEST.COM'} at 127.9.123.65:38213
WARNING: no policy specified for kudu/127.9.123.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
I20260926 15:03:56.853092 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:42715
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:03:57.178280 20275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:57.178577 20275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:57.178697 20275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:57.186123 20275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:57.186244 20275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:57.186396 20275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:03:57.196273 20275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:42715
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:57.197643 20275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:57.199031 20275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:57.212021 20280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:57.212157 20281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:57.212688 20283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:57.212865 20275 server_base.cc:1034] running on GCE node
I20260926 15:03:57.213562 20275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:57.214576 20275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:57.215806 20275 hybrid_clock.cc:648] HybridClock initialized: now 1790435037215751 us; error 42 us; skew 500 ppm
I20260926 15:03:57.216167 20275 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:57 dist-test-slave-7153 krb5kdc[20035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435037, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:57.219339 20275 init.cc:377] Logged in from keytab as kudu/127.9.123.66@KRBTEST.COM (short username kudu)
I20260926 15:03:57.220847 20275 webserver.cc:460] Webserver started at http://127.9.123.66:44265/ using document root <none> and password file <none>
I20260926 15:03:57.221581 20275 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:57.221699 20275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:57.221987 20275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:57.224884 20275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "b131d9d2b25240a5b904077d137210f2"
format_stamp: "Formatted at 2026-09-26 15:03:57 on dist-test-slave-7153"
server_key: "56ba48165a3ee33c70fbb0ed91360f62"
server_key_iv: "aa28a3501c6ac50b6e1518546402b272"
server_key_version: "encryptionkey@0"
I20260926 15:03:57.225699 20275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "b131d9d2b25240a5b904077d137210f2"
format_stamp: "Formatted at 2026-09-26 15:03:57 on dist-test-slave-7153"
server_key: "56ba48165a3ee33c70fbb0ed91360f62"
server_key_iv: "aa28a3501c6ac50b6e1518546402b272"
server_key_version: "encryptionkey@0"
I20260926 15:03:57.231240 20275 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 15:03:57.235246 20290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:57.237004 20275 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 15:03:57.237268 20275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "b131d9d2b25240a5b904077d137210f2"
format_stamp: "Formatted at 2026-09-26 15:03:57 on dist-test-slave-7153"
server_key: "56ba48165a3ee33c70fbb0ed91360f62"
server_key_iv: "aa28a3501c6ac50b6e1518546402b272"
server_key_version: "encryptionkey@0"
I20260926 15:03:57.237512 20275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:57.253305 20275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:57.257478 20275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:57.257757 20275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:57.258747 20275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:57.260145 20275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:57.260224 20275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:57.260344 20275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:57.260401 20275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:57.291075 20275 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:33635
I20260926 15:03:57.291119 20402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:33635 every 8 connection(s)
I20260926 15:03:57.292382 20275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:03:57.295270 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 20275
I20260926 15:03:57.295475 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:03:57.296062 9709 external_mini_cluster.cc:1428] Setting key 7c90623c7014c9165ad19ac7bb1c2548
Sep 26 15:03:57 dist-test-slave-7153 krb5kdc[20035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435037, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:03:57.313778 20271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:57.295137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:47899 (local address 127.9.123.126:42715)
0926 15:03:57.295321 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:57.295327 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:57.296903 (+ 1576us) server_negotiation.cc:407] Connection header received
0926 15:03:57.298789 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:57.298794 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:57.298875 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:57.298976 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:57.300464 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.301386 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.302941 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.303529 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.303649 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:57.307573 (+ 3924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:57.307578 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:57.307581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:57.307616 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:57.310443 (+ 2827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:57.311399 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:57.311403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:57.311406 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:57.311533 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:57.312017 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:57.312020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:57.312022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:57.312264 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:57.312376 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:57.313304 (+ 928us) server_negotiation.cc:299] Negotiation successful
0926 15:03:57.313521 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 15:03:57.315294 20405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:57.295916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:42715 (local address 127.9.123.66:47899)
0926 15:03:57.296727 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:57.296784 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:57.298510 (+ 1726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:57.299159 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:57.299168 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:57.299677 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:57.300295 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:57.300313 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.301539 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.301543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:57.302800 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:57.302811 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.303770 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.303775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:57.304049 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:57.304784 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:57.304808 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:57.307290 (+ 2482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:57.310636 (+ 3346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:57.310647 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:57.310668 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:57.311276 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:57.311635 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:57.311640 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:57.311643 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:57.311912 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:57.312370 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:57.312378 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:57.312836 (+ 458us) client_negotiation.cc:769] Sending connection context
0926 15:03:57.313214 (+ 378us) client_negotiation.cc:240] Negotiation successful
0926 15:03:57.313787 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":311,"threads_started":1}
I20260926 15:03:57.316902 20403 heartbeater.cc:347] Connected to a master server at 127.9.123.126:42715
I20260926 15:03:57.317222 20403 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:57.317988 20403 heartbeater.cc:511] Master 127.9.123.126:42715 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.123.67@KRBTEST.COM; defaulting to no policy
I20260926 15:03:57.319561 20080 ts_manager.cc:194] Registered new tserver with Master: b131d9d2b25240a5b904077d137210f2 (127.9.123.66:33635)
I20260926 15:03:57.320377 20080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.66@KRBTEST.COM'} at 127.9.123.66:47899
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
I20260926 15:03:57.348685 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:42715
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:03:57.678552 20408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:03:57.678846 20408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:03:57.678966 20408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:03:57.686363 20408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:03:57.686517 20408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:03:57.686709 20408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:03:57.696770 20408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:42715
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:03:57.698191 20408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:03:57.699577 20408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:03:57.713376 20416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:57.713420 20414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:03:57.713464 20413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:03:57.714634 20408 server_base.cc:1034] running on GCE node
I20260926 15:03:57.715286 20408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:03:57.716274 20408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:03:57.717487 20408 hybrid_clock.cc:648] HybridClock initialized: now 1790435037717432 us; error 50 us; skew 500 ppm
I20260926 15:03:57.717811 20408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:03:57 dist-test-slave-7153 krb5kdc[20035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435037, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:03:57.721140 20408 init.cc:377] Logged in from keytab as kudu/127.9.123.67@KRBTEST.COM (short username kudu)
I20260926 15:03:57.722803 20408 webserver.cc:460] Webserver started at http://127.9.123.67:40475/ using document root <none> and password file <none>
I20260926 15:03:57.723515 20408 fs_manager.cc:359] Metadata directory not provided
I20260926 15:03:57.723634 20408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:03:57.723929 20408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:03:57.727187 20408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "50371338009543bebb18c8c2d90fd46b"
format_stamp: "Formatted at 2026-09-26 15:03:57 on dist-test-slave-7153"
server_key: "533f7813d24765c4e870c391a2d612a0"
server_key_iv: "1d76aa30c1425adb11999ef92cb069d4"
server_key_version: "encryptionkey@0"
I20260926 15:03:57.728086 20408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "50371338009543bebb18c8c2d90fd46b"
format_stamp: "Formatted at 2026-09-26 15:03:57 on dist-test-slave-7153"
server_key: "533f7813d24765c4e870c391a2d612a0"
server_key_iv: "1d76aa30c1425adb11999ef92cb069d4"
server_key_version: "encryptionkey@0"
I20260926 15:03:57.733783 20408 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 15:03:57.738013 20423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:57.739547 20408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:03:57.739729 20408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "50371338009543bebb18c8c2d90fd46b"
format_stamp: "Formatted at 2026-09-26 15:03:57 on dist-test-slave-7153"
server_key: "533f7813d24765c4e870c391a2d612a0"
server_key_iv: "1d76aa30c1425adb11999ef92cb069d4"
server_key_version: "encryptionkey@0"
I20260926 15:03:57.740056 20408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:03:57.758190 20408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:03:57.763036 20408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:03:57.763365 20408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:03:57.764436 20408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:03:57.765967 20408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:03:57.766050 20408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:57.766168 20408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:03:57.766217 20408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:03:57.797501 20408 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:35447
I20260926 15:03:57.797545 20535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:35447 every 8 connection(s)
I20260926 15:03:57.798749 20408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:03:57.800968 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 20408
I20260926 15:03:57.801174 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:03:57.801681 9709 external_mini_cluster.cc:1428] Setting key 79155239f86d4feec25ae9bb88fc388a
Sep 26 15:03:57 dist-test-slave-7153 krb5kdc[20035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435037, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:03:57.819093 20271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:57.801242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:34209 (local address 127.9.123.126:42715)
0926 15:03:57.801443 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:57.801449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:57.802778 (+ 1329us) server_negotiation.cc:407] Connection header received
0926 15:03:57.804955 (+ 2177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:57.804960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:57.805038 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:57.805136 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:57.806638 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.807373 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.808950 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.809404 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.809524 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:57.813194 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:57.813199 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:57.813203 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:57.813238 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:57.815790 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:57.816733 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:57.816740 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:57.816743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:57.816894 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:57.817407 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:57.817412 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:57.817414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:57.817727 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:57.817847 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:57.818603 (+ 756us) server_negotiation.cc:299] Negotiation successful
0926 15:03:57.818792 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 15:03:57.820014 20538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:57.801834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:42715 (local address 127.9.123.67:34209)
0926 15:03:57.802611 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:57.802662 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:57.804629 (+ 1967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:57.805403 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:57.805412 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:57.805910 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:03:57.806478 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:57.806496 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.807521 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.807527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:57.808805 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:57.808818 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.809639 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.809645 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:57.809887 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:57.810552 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:03:57.810580 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:03:57.812904 (+ 2324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:03:57.815999 (+ 3095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:57.816011 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:57.816033 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:57.816619 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:57.816999 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:03:57.817002 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:03:57.817005 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:03:57.817282 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:03:57.817857 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:03:57.817864 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:03:57.818238 (+ 374us) client_negotiation.cc:769] Sending connection context
0926 15:03:57.818482 (+ 244us) client_negotiation.cc:240] Negotiation successful
0926 15:03:57.818887 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":296,"threads_started":1}
I20260926 15:03:57.821506 20536 heartbeater.cc:347] Connected to a master server at 127.9.123.126:42715
I20260926 15:03:57.821799 20536 heartbeater.cc:464] Registering TS with master...
I20260926 15:03:57.822578 20536 heartbeater.cc:511] Master 127.9.123.126:42715 requested a full tablet report, sending...
I20260926 15:03:57.824193 20080 ts_manager.cc:194] Registered new tserver with Master: 50371338009543bebb18c8c2d90fd46b (127.9.123.67:35447)
I20260926 15:03:57.825320 20080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.67@KRBTEST.COM'} at 127.9.123.67:34209
I20260926 15:03:57.826298 20269 heartbeater.cc:503] Master 127.9.123.126:42715 was elected leader, sending a full tablet report...
I20260926 15:03:57.829568 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 15:03:57.846407 20271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:57.834043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39424 (local address 127.9.123.126:42715)
0926 15:03:57.834260 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:57.834266 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:57.834567 (+ 301us) server_negotiation.cc:407] Connection header received
0926 15:03:57.834712 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:57.834715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:57.834782 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:57.834868 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:57.836262 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.837196 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.838590 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.839011 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.839111 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:57.841238 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:57.841242 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:57.841246 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:57.841279 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:57.843934 (+ 2655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:57.844604 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:57.844607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:57.844610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:57.844741 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:57.845166 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:57.845169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:57.845171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:57.845528 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:57.845707 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:57.845969 (+ 262us) server_negotiation.cc:299] Negotiation successful
0926 15:03:57.846127 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 15:03:57.850342 20080 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39424:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:03:57.857218 20080 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:03:57.893379 20551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:57.871196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36732 (local address 127.9.123.65:33299)
0926 15:03:57.882589 (+ 11393us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:57.882602 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:57.883870 (+ 1268us) server_negotiation.cc:407] Connection header received
0926 15:03:57.884675 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:57.884682 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:57.884944 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:57.885139 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:57.886333 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.887640 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.890080 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.891585 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.891741 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:57.892139 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:57.892804 (+ 665us) server_negotiation.cc:299] Negotiation successful
0926 15:03:57.893004 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11205,"thread_start_us":11047,"threads_started":1}
I20260926 15:03:57.893566 20548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:57.870665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:33299 (local address 127.0.0.1:36732)
0926 15:03:57.883678 (+ 13013us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:57.883726 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:57.884484 (+ 758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:57.885415 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:57.885422 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:57.885467 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:57.886115 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:57.886124 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.887803 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.887826 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:57.889906 (+ 2080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:57.889921 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.892308 (+ 2387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.892316 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:57.892527 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:57.892582 (+ 55us) client_negotiation.cc:769] Sending connection context
0926 15:03:57.892716 (+ 134us) client_negotiation.cc:240] Negotiation successful
0926 15:03:57.893001 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12742,"mutex_wait_us":7,"spinlock_wait_cycles":2304,"thread_start_us":12580,"threads_started":1}
I20260926 15:03:57.893934 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:57.871538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45142 (local address 127.9.123.67:35447)
0926 15:03:57.883070 (+ 11532us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:57.883079 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:57.883875 (+ 796us) server_negotiation.cc:407] Connection header received
0926 15:03:57.884668 (+ 793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:57.884677 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:57.884947 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:57.885142 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:57.886037 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.887624 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.890652 (+ 3028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.892330 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.892494 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:57.892729 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:57.893307 (+ 578us) server_negotiation.cc:299] Negotiation successful
0926 15:03:57.893630 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11377,"thread_start_us":11246,"threads_started":1}
I20260926 15:03:57.894141 20552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:57.871417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46874 (local address 127.9.123.66:33635)
0926 15:03:57.884263 (+ 12846us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:57.884272 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:57.884389 (+ 117us) server_negotiation.cc:407] Connection header received
0926 15:03:57.884673 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:57.884682 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:57.884944 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:57.885169 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:57.886237 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.888426 (+ 2189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.890941 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.892421 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.892599 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:57.892787 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:57.893674 (+ 887us) server_negotiation.cc:299] Negotiation successful
0926 15:03:57.893840 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12709,"thread_start_us":12632,"threads_started":1}
I20260926 15:03:57.894636 20549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:57.871172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:33635 (local address 127.0.0.1:46874)
0926 15:03:57.884277 (+ 13105us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:57.884292 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:57.884419 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:57.885395 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:57.885400 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:57.885428 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:57.886059 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:57.886073 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.888591 (+ 2518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.888596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:57.890792 (+ 2196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:57.890802 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.892962 (+ 2160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.892969 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:57.893174 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:57.893208 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 15:03:57.893284 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 15:03:57.893436 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12991,"mutex_wait_us":27,"thread_start_us":12263,"threads_started":1}
I20260926 15:03:57.894737 20550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:57.870812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:35447 (local address 127.0.0.1:45142)
0926 15:03:57.883705 (+ 12893us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:57.883725 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:57.884416 (+ 691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:57.885316 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:57.885323 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:57.885365 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:57.885858 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:57.885868 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.887783 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.887789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:57.890458 (+ 2669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:57.890477 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.892874 (+ 2397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.892880 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:57.893060 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:57.893091 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 15:03:57.894187 (+ 1096us) client_negotiation.cc:240] Negotiation successful
0926 15:03:57.894340 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12759,"mutex_wait_us":147,"thread_start_us":13222,"threads_started":1}
I20260926 15:03:57.897619 20204 tablet_service.cc:1467] Processing CreateTablet for tablet e060f66807344d70aa496bbab3458887 (DEFAULT_TABLE table=test-table [id=61a2cae7187d4a3ab387adcd502409eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:03:57.897810 20471 tablet_service.cc:1467] Processing CreateTablet for tablet e060f66807344d70aa496bbab3458887 (DEFAULT_TABLE table=test-table [id=61a2cae7187d4a3ab387adcd502409eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:03:57.899206 20471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e060f66807344d70aa496bbab3458887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:57.899398 20204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e060f66807344d70aa496bbab3458887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:57.899858 20338 tablet_service.cc:1467] Processing CreateTablet for tablet e060f66807344d70aa496bbab3458887 (DEFAULT_TABLE table=test-table [id=61a2cae7187d4a3ab387adcd502409eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:03:57.901337 20338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e060f66807344d70aa496bbab3458887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:03:57.912472 20554 tablet_bootstrap.cc:492] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209: Bootstrap starting.
I20260926 15:03:57.915808 20554 tablet_bootstrap.cc:654] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:57.915967 20556 tablet_bootstrap.cc:492] T e060f66807344d70aa496bbab3458887 P b131d9d2b25240a5b904077d137210f2: Bootstrap starting.
I20260926 15:03:57.917685 20555 tablet_bootstrap.cc:492] T e060f66807344d70aa496bbab3458887 P 50371338009543bebb18c8c2d90fd46b: Bootstrap starting.
I20260926 15:03:57.918018 20554 log.cc:824] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:57.920372 20556 tablet_bootstrap.cc:654] T e060f66807344d70aa496bbab3458887 P b131d9d2b25240a5b904077d137210f2: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:57.920570 20555 tablet_bootstrap.cc:654] T e060f66807344d70aa496bbab3458887 P 50371338009543bebb18c8c2d90fd46b: Neither blocks nor log segments found. Creating new log.
I20260926 15:03:57.921546 20556 log.cc:824] T e060f66807344d70aa496bbab3458887 P b131d9d2b25240a5b904077d137210f2: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:57.921806 20555 log.cc:824] T e060f66807344d70aa496bbab3458887 P 50371338009543bebb18c8c2d90fd46b: Log is configured to *not* fsync() on all Append() calls
I20260926 15:03:57.923331 20554 tablet_bootstrap.cc:492] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209: No bootstrap required, opened a new log
I20260926 15:03:57.923645 20554 ts_tablet_manager.cc:1397] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260926 15:03:57.924572 20556 tablet_bootstrap.cc:492] T e060f66807344d70aa496bbab3458887 P b131d9d2b25240a5b904077d137210f2: No bootstrap required, opened a new log
I20260926 15:03:57.924839 20556 ts_tablet_manager.cc:1397] T e060f66807344d70aa496bbab3458887 P b131d9d2b25240a5b904077d137210f2: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 15:03:57.927450 20555 tablet_bootstrap.cc:492] T e060f66807344d70aa496bbab3458887 P 50371338009543bebb18c8c2d90fd46b: No bootstrap required, opened a new log
I20260926 15:03:57.927704 20555 ts_tablet_manager.cc:1397] T e060f66807344d70aa496bbab3458887 P 50371338009543bebb18c8c2d90fd46b: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.002s
I20260926 15:03:57.934430 20554 raft_consensus.cc:348] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d229a50bfc594523a5ec08e4b1a0d209" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33299 } } peers { permanent_uuid: "50371338009543bebb18c8c2d90fd46b" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35447 } } peers { permanent_uuid: "b131d9d2b25240a5b904077d137210f2" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33635 } }
I20260926 15:03:57.934954 20554 raft_consensus.cc:374] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:57.934820 20555 raft_consensus.cc:348] T e060f66807344d70aa496bbab3458887 P 50371338009543bebb18c8c2d90fd46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d229a50bfc594523a5ec08e4b1a0d209" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33299 } } peers { permanent_uuid: "50371338009543bebb18c8c2d90fd46b" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35447 } } peers { permanent_uuid: "b131d9d2b25240a5b904077d137210f2" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33635 } }
I20260926 15:03:57.935091 20554 raft_consensus.cc:729] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d229a50bfc594523a5ec08e4b1a0d209, State: Initialized, Role: FOLLOWER
I20260926 15:03:57.935226 20555 raft_consensus.cc:374] T e060f66807344d70aa496bbab3458887 P 50371338009543bebb18c8c2d90fd46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:57.935328 20555 raft_consensus.cc:729] T e060f66807344d70aa496bbab3458887 P 50371338009543bebb18c8c2d90fd46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50371338009543bebb18c8c2d90fd46b, State: Initialized, Role: FOLLOWER
I20260926 15:03:57.935669 20556 raft_consensus.cc:348] T e060f66807344d70aa496bbab3458887 P b131d9d2b25240a5b904077d137210f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d229a50bfc594523a5ec08e4b1a0d209" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33299 } } peers { permanent_uuid: "50371338009543bebb18c8c2d90fd46b" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35447 } } peers { permanent_uuid: "b131d9d2b25240a5b904077d137210f2" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33635 } }
I20260926 15:03:57.935771 20554 consensus_queue.cc:260] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d229a50bfc594523a5ec08e4b1a0d209" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33299 } } peers { permanent_uuid: "50371338009543bebb18c8c2d90fd46b" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35447 } } peers { permanent_uuid: "b131d9d2b25240a5b904077d137210f2" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33635 } }
I20260926 15:03:57.935961 20555 consensus_queue.cc:260] T e060f66807344d70aa496bbab3458887 P 50371338009543bebb18c8c2d90fd46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d229a50bfc594523a5ec08e4b1a0d209" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33299 } } peers { permanent_uuid: "50371338009543bebb18c8c2d90fd46b" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35447 } } peers { permanent_uuid: "b131d9d2b25240a5b904077d137210f2" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33635 } }
I20260926 15:03:57.936115 20556 raft_consensus.cc:374] T e060f66807344d70aa496bbab3458887 P b131d9d2b25240a5b904077d137210f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:03:57.936214 20556 raft_consensus.cc:729] T e060f66807344d70aa496bbab3458887 P b131d9d2b25240a5b904077d137210f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b131d9d2b25240a5b904077d137210f2, State: Initialized, Role: FOLLOWER
I20260926 15:03:57.936930 20556 consensus_queue.cc:260] T e060f66807344d70aa496bbab3458887 P b131d9d2b25240a5b904077d137210f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d229a50bfc594523a5ec08e4b1a0d209" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33299 } } peers { permanent_uuid: "50371338009543bebb18c8c2d90fd46b" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35447 } } peers { permanent_uuid: "b131d9d2b25240a5b904077d137210f2" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33635 } }
I20260926 15:03:57.937688 20554 ts_tablet_manager.cc:1428] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209: Time spent starting tablet: real 0.014s user 0.005s sys 0.009s
I20260926 15:03:57.939393 20536 heartbeater.cc:503] Master 127.9.123.126:42715 was elected leader, sending a full tablet report...
I20260926 15:03:57.939725 20555 ts_tablet_manager.cc:1428] T e060f66807344d70aa496bbab3458887 P 50371338009543bebb18c8c2d90fd46b: Time spent starting tablet: real 0.012s user 0.008s sys 0.003s
I20260926 15:03:57.940095 20556 ts_tablet_manager.cc:1428] T e060f66807344d70aa496bbab3458887 P b131d9d2b25240a5b904077d137210f2: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260926 15:03:57.940630 20403 heartbeater.cc:503] Master 127.9.123.126:42715 was elected leader, sending a full tablet report...
I20260926 15:03:57.943593 20560 raft_consensus.cc:482] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:03:57.943857 20560 raft_consensus.cc:504] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d229a50bfc594523a5ec08e4b1a0d209" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33299 } } peers { permanent_uuid: "50371338009543bebb18c8c2d90fd46b" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35447 } } peers { permanent_uuid: "b131d9d2b25240a5b904077d137210f2" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33635 } }
I20260926 15:03:57.945665 20560 leader_election.cc:288] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50371338009543bebb18c8c2d90fd46b (127.9.123.67:35447), b131d9d2b25240a5b904077d137210f2 (127.9.123.66:33635)
I20260926 15:03:57.953087 20552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:57.946354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:38767 (local address 127.9.123.66:33635)
0926 15:03:57.946532 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:57.946538 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:57.946755 (+ 217us) server_negotiation.cc:407] Connection header received
0926 15:03:57.946896 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:57.946901 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:57.946972 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:57.947068 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:57.947500 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.948716 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.950721 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.952071 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.952263 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:57.952331 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:57.952658 (+ 327us) server_negotiation.cc:299] Negotiation successful
0926 15:03:57.952791 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 15:03:57.953152 20564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:57.945905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:33635 (local address 127.9.123.65:38767)
0926 15:03:57.946670 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:57.946684 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:57.946791 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:57.947078 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:57.947081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:57.947099 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:57.947383 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:57.947390 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.948922 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.948928 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:57.950486 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:57.950503 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.952317 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.952322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:57.952472 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:57.952500 (+ 28us) client_negotiation.cc:769] Sending connection context
0926 15:03:57.952603 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 15:03:57.952755 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"threads_started":1}
I20260926 15:03:57.954710 20358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e060f66807344d70aa496bbab3458887" candidate_uuid: "d229a50bfc594523a5ec08e4b1a0d209" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b131d9d2b25240a5b904077d137210f2" is_pre_election: true
I20260926 15:03:57.955197 20358 raft_consensus.cc:2454] T e060f66807344d70aa496bbab3458887 P b131d9d2b25240a5b904077d137210f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d229a50bfc594523a5ec08e4b1a0d209 in term 0.
I20260926 15:03:57.955873 20158 leader_election.cc:302] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b131d9d2b25240a5b904077d137210f2, d229a50bfc594523a5ec08e4b1a0d209; no voters:
I20260926 15:03:57.956298 20560 raft_consensus.cc:2781] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:03:57.956441 20560 raft_consensus.cc:482] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:03:57.956559 20560 raft_consensus.cc:3037] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:57.958709 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:57.945891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:47173 (local address 127.9.123.67:35447)
0926 15:03:57.946107 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:57.946112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:57.953504 (+ 7392us) server_negotiation.cc:407] Connection header received
0926 15:03:57.953811 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:57.953815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:57.953879 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:57.953966 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:03:57.954420 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.955187 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.957039 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.957872 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.958022 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:57.958094 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:57.958346 (+ 252us) server_negotiation.cc:299] Negotiation successful
0926 15:03:57.958475 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260926 15:03:57.958729 20564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:57.945998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:35447 (local address 127.9.123.65:47173)
0926 15:03:57.953374 (+ 7376us) negotiation.cc:107] Waiting for socket to connect
0926 15:03:57.953418 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 15:03:57.953628 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:03:57.954119 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:03:57.954124 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:03:57.954151 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:03:57.954274 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:57.954282 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.955304 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.955310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:57.956882 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:03:57.956903 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.958022 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.958027 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:03:57.958213 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:57.958235 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 15:03:57.958311 (+ 76us) client_negotiation.cc:240] Negotiation successful
0926 15:03:57.958451 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7253,"thread_start_us":375,"threads_started":1}
I20260926 15:03:57.958876 20560 raft_consensus.cc:504] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d229a50bfc594523a5ec08e4b1a0d209" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33299 } } peers { permanent_uuid: "50371338009543bebb18c8c2d90fd46b" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35447 } } peers { permanent_uuid: "b131d9d2b25240a5b904077d137210f2" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33635 } }
I20260926 15:03:57.959618 20560 leader_election.cc:288] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209 [CANDIDATE]: Term 1 election: Requested vote from peers 50371338009543bebb18c8c2d90fd46b (127.9.123.67:35447), b131d9d2b25240a5b904077d137210f2 (127.9.123.66:33635)
I20260926 15:03:57.959652 20491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e060f66807344d70aa496bbab3458887" candidate_uuid: "d229a50bfc594523a5ec08e4b1a0d209" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50371338009543bebb18c8c2d90fd46b" is_pre_election: true
I20260926 15:03:57.960026 20490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e060f66807344d70aa496bbab3458887" candidate_uuid: "d229a50bfc594523a5ec08e4b1a0d209" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50371338009543bebb18c8c2d90fd46b"
I20260926 15:03:57.960083 20358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e060f66807344d70aa496bbab3458887" candidate_uuid: "d229a50bfc594523a5ec08e4b1a0d209" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b131d9d2b25240a5b904077d137210f2"
I20260926 15:03:57.960270 20358 raft_consensus.cc:3037] T e060f66807344d70aa496bbab3458887 P b131d9d2b25240a5b904077d137210f2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:57.960275 20491 raft_consensus.cc:2454] T e060f66807344d70aa496bbab3458887 P 50371338009543bebb18c8c2d90fd46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d229a50bfc594523a5ec08e4b1a0d209 in term 0.
I20260926 15:03:57.962563 20358 raft_consensus.cc:2454] T e060f66807344d70aa496bbab3458887 P b131d9d2b25240a5b904077d137210f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d229a50bfc594523a5ec08e4b1a0d209 in term 1.
I20260926 15:03:57.962922 20158 leader_election.cc:302] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b131d9d2b25240a5b904077d137210f2, d229a50bfc594523a5ec08e4b1a0d209; no voters: 50371338009543bebb18c8c2d90fd46b
I20260926 15:03:57.963195 20560 raft_consensus.cc:2781] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:03:57.963608 20560 raft_consensus.cc:686] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209 [term 1 LEADER]: Becoming Leader. State: Replica: d229a50bfc594523a5ec08e4b1a0d209, State: Running, Role: LEADER
I20260926 15:03:57.964078 20560 consensus_queue.cc:237] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d229a50bfc594523a5ec08e4b1a0d209" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33299 } } peers { permanent_uuid: "50371338009543bebb18c8c2d90fd46b" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35447 } } peers { permanent_uuid: "b131d9d2b25240a5b904077d137210f2" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33635 } }
I20260926 15:03:57.968801 20079 catalog_manager.cc:5495] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209 reported cstate change: term changed from 0 to 1, leader changed from <none> to d229a50bfc594523a5ec08e4b1a0d209 (127.9.123.65). New cstate: current_term: 1 leader_uuid: "d229a50bfc594523a5ec08e4b1a0d209" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d229a50bfc594523a5ec08e4b1a0d209" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50371338009543bebb18c8c2d90fd46b" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b131d9d2b25240a5b904077d137210f2" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33635 } health_report { overall_health: UNKNOWN } } }
I20260926 15:03:58.001443 20551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:57.996302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36746 (local address 127.9.123.65:33299)
0926 15:03:57.996504 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:57.996510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:57.996593 (+ 83us) server_negotiation.cc:407] Connection header received
0926 15:03:57.996802 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:57.996807 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:57.996883 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:57.997049 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:03:57.997406 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.998113 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.999422 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:57.999817 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:57.999922 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:58.000374 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:03:58.000883 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:03:58.001027 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:58.001122 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 15:03:58.001200 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 15:03:58.009073 20491 raft_consensus.cc:3037] T e060f66807344d70aa496bbab3458887 P 50371338009543bebb18c8c2d90fd46b [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:03:58.009343 20358 raft_consensus.cc:1261] T e060f66807344d70aa496bbab3458887 P b131d9d2b25240a5b904077d137210f2 [term 1 FOLLOWER]: Refusing update from remote peer d229a50bfc594523a5ec08e4b1a0d209: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:03:58.010180 20565 consensus_queue.cc:1035] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b131d9d2b25240a5b904077d137210f2" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:03:58.011709 20491 raft_consensus.cc:1261] T e060f66807344d70aa496bbab3458887 P 50371338009543bebb18c8c2d90fd46b [term 1 FOLLOWER]: Refusing update from remote peer d229a50bfc594523a5ec08e4b1a0d209: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:03:58.012428 20565 consensus_queue.cc:1035] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50371338009543bebb18c8c2d90fd46b" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:03:58.021082 20569 mvcc.cc:205] Tried to move back new op lower bound from 7333621915676495872 to 7333621915504488448. Current Snapshot: MvccSnapshot[applied={T|T < 7333621915676495872}]
I20260926 15:03:58.021433 20568 mvcc.cc:205] Tried to move back new op lower bound from 7333621915676495872 to 7333621915504488448. Current Snapshot: MvccSnapshot[applied={T|T < 7333621915676495872}]
I20260926 15:03:58.044945 20080 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:03:58.045320 20080 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:03:58.049016 20080 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8435e35ba83a41aba628b9304a4eaff0: Sending DeleteTablet for 3 replicas of tablet e060f66807344d70aa496bbab3458887
I20260926 15:03:58.050289 20471 tablet_service.cc:1514] Processing DeleteTablet for tablet e060f66807344d70aa496bbab3458887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:03:58 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:45142
I20260926 15:03:58.051142 20578 tablet_replica.cc:330] T e060f66807344d70aa496bbab3458887 P 50371338009543bebb18c8c2d90fd46b: stopping tablet replica
I20260926 15:03:58.051496 20578 raft_consensus.cc:2229] T e060f66807344d70aa496bbab3458887 P 50371338009543bebb18c8c2d90fd46b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:03:58.051535 20338 tablet_service.cc:1514] Processing DeleteTablet for tablet e060f66807344d70aa496bbab3458887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:03:58 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:46874
I20260926 15:03:58.051939 20578 raft_consensus.cc:2258] T e060f66807344d70aa496bbab3458887 P 50371338009543bebb18c8c2d90fd46b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:58.052618 20579 tablet_replica.cc:330] T e060f66807344d70aa496bbab3458887 P b131d9d2b25240a5b904077d137210f2: stopping tablet replica
I20260926 15:03:58.052630 20204 tablet_service.cc:1514] Processing DeleteTablet for tablet e060f66807344d70aa496bbab3458887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:03:58 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:36732
I20260926 15:03:58.053061 20579 raft_consensus.cc:2229] T e060f66807344d70aa496bbab3458887 P b131d9d2b25240a5b904077d137210f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:03:58.053584 20579 raft_consensus.cc:2258] T e060f66807344d70aa496bbab3458887 P b131d9d2b25240a5b904077d137210f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:58.053951 20578 ts_tablet_manager.cc:1905] T e060f66807344d70aa496bbab3458887 P 50371338009543bebb18c8c2d90fd46b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:03:58.054709 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 20141
I20260926 15:03:58.055572 20579 ts_tablet_manager.cc:1905] T e060f66807344d70aa496bbab3458887 P b131d9d2b25240a5b904077d137210f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:03:58.060742 20578 ts_tablet_manager.cc:1918] T e060f66807344d70aa496bbab3458887 P 50371338009543bebb18c8c2d90fd46b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:03:58.060949 20578 log.cc:1196] T e060f66807344d70aa496bbab3458887 P 50371338009543bebb18c8c2d90fd46b: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/e060f66807344d70aa496bbab3458887
I20260926 15:03:58.061514 20578 ts_tablet_manager.cc:1939] T e060f66807344d70aa496bbab3458887 P 50371338009543bebb18c8c2d90fd46b: Deleting consensus metadata
I20260926 15:03:58.061633 20580 tablet_replica.cc:330] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209: stopping tablet replica
I20260926 15:03:58.062111 20580 raft_consensus.cc:2229] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209 [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:03:58.062770 20580 raft_consensus.cc:2258] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:03:58.063373 20067 catalog_manager.cc:4841] TS 50371338009543bebb18c8c2d90fd46b (127.9.123.67:35447): tablet e060f66807344d70aa496bbab3458887 (table test-table [id=61a2cae7187d4a3ab387adcd502409eb]) successfully deleted
I20260926 15:03:58.064700 20580 ts_tablet_manager.cc:1905] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:03:58.070170 20580 ts_tablet_manager.cc:1918] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:03:58.070475 20580 log.cc:1196] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-0/wal/wals/e060f66807344d70aa496bbab3458887
I20260926 15:03:58.070825 20579 ts_tablet_manager.cc:1918] T e060f66807344d70aa496bbab3458887 P b131d9d2b25240a5b904077d137210f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:03:58.071053 20579 log.cc:1196] T e060f66807344d70aa496bbab3458887 P b131d9d2b25240a5b904077d137210f2: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/e060f66807344d70aa496bbab3458887
I20260926 15:03:58.071116 20580 ts_tablet_manager.cc:1939] T e060f66807344d70aa496bbab3458887 P d229a50bfc594523a5ec08e4b1a0d209: Deleting consensus metadata
I20260926 15:03:58.071785 20579 ts_tablet_manager.cc:1939] T e060f66807344d70aa496bbab3458887 P b131d9d2b25240a5b904077d137210f2: Deleting consensus metadata
I20260926 15:03:58.073244 20064 catalog_manager.cc:4841] TS d229a50bfc594523a5ec08e4b1a0d209 (127.9.123.65:33299): tablet e060f66807344d70aa496bbab3458887 (table test-table [id=61a2cae7187d4a3ab387adcd502409eb]) successfully deleted
I20260926 15:03:58.073783 20065 catalog_manager.cc:4841] TS b131d9d2b25240a5b904077d137210f2 (127.9.123.66:33635): tablet e060f66807344d70aa496bbab3458887 (table test-table [id=61a2cae7187d4a3ab387adcd502409eb]) successfully deleted
Sep 26 15:03:58 dist-test-slave-7153 krb5kdc[20035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.65@KRBTEST.COM
I20260926 15:03:58.085788 20551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:58.057219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36760 (local address 127.9.123.65:33299)
0926 15:03:58.057390 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:58.057396 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:58.069687 (+ 12291us) server_negotiation.cc:407] Connection header received
0926 15:03:58.069990 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:58.069996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:58.070090 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:58.070168 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:58.072160 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:58.073306 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:58.074677 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:58.075148 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:58.075272 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:58.079267 (+ 3995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:58.079283 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:58.079299 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:58.079348 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:58.082665 (+ 3317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:58.083469 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:58.083477 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:58.083480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:58.083647 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:58.084138 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:58.084144 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:58.084147 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:58.084564 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:58.084902 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:58.085206 (+ 304us) server_negotiation.cc:299] Negotiation successful
0926 15:03:58.085442 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 15:03:58.086552 20264 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:58.183955 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 20141
I20260926 15:03:58.201784 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 20275
Sep 26 15:03:58 dist-test-slave-7153 krb5kdc[20035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.66@KRBTEST.COM
I20260926 15:03:58.217572 20552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:58.202571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46882 (local address 127.9.123.66:33635)
0926 15:03:58.202744 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:58.202750 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:58.202770 (+ 20us) server_negotiation.cc:407] Connection header received
0926 15:03:58.202833 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:58.202838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:58.202917 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:58.203022 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:58.204149 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:58.205366 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:58.206643 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:58.207255 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:58.207367 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:58.211257 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:58.211276 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:58.211298 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:58.211347 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:58.214884 (+ 3537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:58.215592 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:58.215602 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:58.215606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:58.215749 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:58.216094 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:58.216099 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:58.216103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:58.216572 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:58.216842 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:58.217125 (+ 283us) server_negotiation.cc:299] Negotiation successful
0926 15:03:58.217317 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 15:03:58.218264 20398 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:58.306022 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 20275
I20260926 15:03:58.319499 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 20408
Sep 26 15:03:58 dist-test-slave-7153 krb5kdc[20035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.67@KRBTEST.COM
I20260926 15:03:58.333935 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:03:58.320279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45148 (local address 127.9.123.67:35447)
0926 15:03:58.320443 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 15:03:58.320447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:03:58.320466 (+ 19us) server_negotiation.cc:407] Connection header received
0926 15:03:58.320656 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:03:58.320660 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:03:58.320729 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:03:58.320809 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:03:58.322037 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:58.322761 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:58.324072 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:03:58.324496 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:03:58.324605 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:03:58.328866 (+ 4261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:03:58.328882 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:03:58.328899 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:03:58.328936 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:03:58.331512 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:58.332190 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:58.332198 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:58.332200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:58.332338 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:03:58.332609 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:03:58.332616 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:03:58.332618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:03:58.332953 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:03:58.333239 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0926 15:03:58.333447 (+ 208us) server_negotiation.cc:299] Negotiation successful
0926 15:03:58.333627 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 15:03:58.334573 20531 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:58.423700 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 20408
I20260926 15:03:58.436534 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 20048
I20260926 15:03:58.437733 20110 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:03:58.561491 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 20048
2026-09-26T15:03:58Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4069 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:03:58 dist-test-slave-7153 krb5kdc[20588](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:03:58 dist-test-slave-7153 krb5kdc[20588](info): set up 2 sockets
Sep 26 15:03:58 dist-test-slave-7153 krb5kdc[20588](info): commencing operation
krb5kdc: starting...
W20260926 15:03:59.593847 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.994s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:03:59 dist-test-slave-7153 krb5kdc[20588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:03:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:03:59Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.123.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.123.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.123.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/rangeradmin_127.9.123.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.123.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/rangeradmin_127.9.123.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.123.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.123.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.123.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/rangerlookup_127.9.123.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.123.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/rangerlookup_127.9.123.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.123.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.123.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.123.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/HTTP_127.9.123.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.123.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/HTTP_127.9.123.84@KRBTEST.COM.keytab.
I20260926 15:03:59.913647 9709 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskR962dp/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/postgres -l logfile start
2026-09-26 15:04:02.566 UTC [20618] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 15:04:02.566 UTC [20618] LOG: listening on IPv4 address "127.9.123.84", port 45825
2026-09-26 15:04:02.570 UTC [20618] LOG: listening on Unix socket "/tmp/.s.PGSQL.45825"
2026-09-26 15:04:02.592 UTC [20621] LOG: database system was shut down at 2026-09-26 15:04:00 UTC
2026-09-26 15:04:02.596 UTC [20618] LOG: database system is ready to accept connections
I20260926 15:04:03.493476 9709 mini_postgres.cc:96] Postgres bound to 45825
2026-09-26 15:04:03.510 UTC [20629] FATAL: database "slave" does not exist
127.9.123.84:45825 - accepting connections
I20260926 15:04:03.511070 9709 mini_ranger.cc:161] Starting Ranger...
I20260926 15:04:03.537684 9709 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 15:04:04.089445 9709 mini_ranger.cc:87] Created ranger Postgres database
I20260926 15:04:04.089597 9709 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ranger-admin
2026-09-26 15:04:04,379 [I] DB FLAVOR :POSTGRES
2026-09-26 15:04:04,380 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 15:04:04,380 [I] Checking connection..
2026-09-26 15:04:04,380 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:45825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 15:04:04,637 [I] Checking connection passed.
2026-09-26 15:04:04,637 [I] --------- Verifying version history table ---------
2026-09-26 15:04:04,637 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:45825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 15:04:04,882 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 15:04:04,882 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:45825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 15:04:05,149 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 15:04:05,150 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 15:04:05,150 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:45825/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 15:04:05.414 UTC [20693] WARNING: there is no transaction in progress
2026-09-26 15:04:05,420 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 15:04:05,420 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:45825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 15:04:05,661 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 15:04:05,661 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 15:04:05,662 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:45825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 15:04:05,905 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:45825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 15:04:06,140 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:45825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7153.c.gcp-upstream.internal','N') ;"
2026-09-26 15:04:06,388 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:45825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 15:04:06,646 [I] Table x_portal_user does not exist in database ranger
2026-09-26 15:04:06,646 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:45825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 15:04:06,889 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 15:04:06,889 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:45825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 15:04:07,154 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:45825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 15:04:07,419 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 15:04:07,420 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:45825/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 15:04:08.469 UTC [20801] WARNING: there is no transaction in progress
2026-09-26 15:04:08.488 UTC [20801] WARNING: there is no transaction in progress
2026-09-26 15:04:08.510 UTC [20801] WARNING: there is no transaction in progress
2026-09-26 15:04:08.530 UTC [20801] WARNING: there is no transaction in progress
2026-09-26 15:04:08.552 UTC [20801] WARNING: there is no transaction in progress
2026-09-26 15:04:08.573 UTC [20801] WARNING: there is no transaction in progress
2026-09-26 15:04:08.636 UTC [20801] WARNING: there is no transaction in progress
2026-09-26 15:04:08.644 UTC [20801] WARNING: there is no transaction in progress
2026-09-26 15:04:08.660 UTC [20801] WARNING: there is no transaction in progress
2026-09-26 15:04:08.672 UTC [20801] WARNING: there is no transaction in progress
2026-09-26 15:04:08.683 UTC [20801] WARNING: there is no transaction in progress
2026-09-26 15:04:08.699 UTC [20801] WARNING: there is no transaction in progress
2026-09-26 15:04:08.711 UTC [20801] WARNING: there is no transaction in progress
2026-09-26 15:04:08.722 UTC [20801] WARNING: there is no transaction in progress
2026-09-26 15:04:08.731 UTC [20801] WARNING: there is no transaction in progress
2026-09-26 15:04:08.740 UTC [20801] WARNING: there is no transaction in progress
2026-09-26 15:04:09,556 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 15:04:09,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:45825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 15:04:09,819 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 15:04:09,820 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:45825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 15:04:10,089 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 15:04:10,090 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:45825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 15:04:10,356 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 15:04:10,357 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:45825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 15:04:10,624 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:45825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 15:04:10,859 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:45825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7153.c.gcp-upstream.internal';"
2026-09-26 15:04:11,097 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 15:04:11,098 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:45825/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 15:04:11,356 [I] DB_PATCHES have already been applied
I20260926 15:04:11.362026 9709 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ranger-admin:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskR962dp/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 15:04:11.362119 9709 mini_ranger.cc:193] Using host: 127.9.123.84
I20260926 15:04:11.376363 9709 mini_ranger.cc:227] Ranger admin URL: http://127.9.123.84:48643
Sep 26, 2026 3:04:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 3:04:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 3:04:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 3:04:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 3:04:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 3:04:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 3:04:12 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 15:04:12.434444 9709 mini_ranger.cc:160] Time spent starting Ranger: real 8.923s user 0.000s sys 0.043s
Sep 26, 2026 3:04:33 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 3:04:33 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 26, 2026 3:04:33 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 3:04:33 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 3:04:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 3:04:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 3:04:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 3:04:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 3:04:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 3:04:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 3:04:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 3:04:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 3:04:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 3:04:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 3:04:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 3:04:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 3:04:33 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 3:04:34 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 15:04:35.105139 9709 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.123.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.123.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.123.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/rangerkms_127.9.123.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.123.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/rangerkms_127.9.123.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.123.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.123.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.123.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/HTTP_127.9.123.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.123.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/HTTP_127.9.123.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 15:04:35 dist-test-slave-7153 krb5kdc[20588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435075, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 15:04:35.289402 9709 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 15:04:35.315299 9709 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 15:04:36.302644 9709 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 15:04:36.302802 9709 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ranger-kms
I20260926 15:04:36.302935 9709 mini_ranger_kms.cc:227] Using postgres at 127.9.123.84:45825
2026-09-26 15:04:36,696 [I] DB FLAVOR :POSTGRES
2026-09-26 15:04:36,696 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 15:04:36,696 [I] Checking connection
2026-09-26 15:04:36,696 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:45825/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 15:04:36,931 [I] connection success
2026-09-26 15:04:36,931 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 15:04:36,931 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 15:04:36,931 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:45825/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 15:04:37,208 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 15:04:37,208 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 15:04:37,208 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 15:04:37,208 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:45825/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 15:04:37,477 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 15:04:38.005048 9709 mini_ranger_kms.cc:310] Created kms service
I20260926 15:04:38.223829 9709 mini_ranger_kms.cc:326] Created kudu user
I20260926 15:04:38.271559 9709 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 15:04:38 dist-test-slave-7153 krb5kdc[20588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435075, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.123.84@KRBTEST.COM
I20260926 15:04:38.720400 9709 mini_ranger_kms.cc:384] Added ranger policy
I20260926 15:04:38.720821 9709 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ranger-kms:/tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 15:04:38.720887 9709 mini_ranger_kms.cc:240] Using host: 127.9.123.84
I20260926 15:04:38.738183 9709 mini_ranger_kms.cc:276] Ranger KMS PID: 20982
I20260926 15:04:38.738386 9709 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.123.84:57103
Sep 26, 2026 3:04:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 3:04:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 3:04:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 3:04:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 3:04:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 3:04:40 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 15:04:40.835211 9709 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.546s user 0.111s sys 0.305s
I20260926 15:04:40.835457 9709 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 15:04:40.835668 9709 mini_ranger_kms.cc:401] 127.9.123.84:57103/kms/v1/keys
Sep 26 15:04:40 dist-test-slave-7153 krb5kdc[20588](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790435080, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.123.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 15:04:41 dist-test-slave-7153 krb5kdc[20588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435080, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.123.84@KRBTEST.COM for HTTP/127.9.123.84@KRBTEST.COM
Sep 26 15:04:41 dist-test-slave-7153 krb5kdc[20588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435080, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.123.84@KRBTEST.COM for HTTP/127.9.123.84@KRBTEST.COM
Sep 26 15:04:44 dist-test-slave-7153 krb5kdc[20588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:04:44.157390 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:33629
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:38057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:33629
--ranger_config_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.123.84:57103/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:04:44.520488 21032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:04:44.520792 21032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:04:44.520879 21032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:04:44.528484 21032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:04:44.528626 21032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:04:44.528695 21032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:04:44.528784 21032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:04:44.528836 21032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:04:44.540274 21032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:38057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.123.84:57103/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:33629
--ranger_config_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:33629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:04:44.541720 21032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:04:44.543110 21032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:04:44.557687 21038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:04:44.557690 21037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:04:44.557719 21040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:04:44.558431 21032 server_base.cc:1034] running on GCE node
I20260926 15:04:44.559296 21032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:04:44.561033 21032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:04:44.562366 21032 hybrid_clock.cc:648] HybridClock initialized: now 1790435084562332 us; error 62 us; skew 500 ppm
I20260926 15:04:44.562764 21032 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:04:44 dist-test-slave-7153 krb5kdc[20588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435084, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:04:44.572815 21032 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:04:44.574641 21032 webserver.cc:460] Webserver started at http://127.9.123.126:35413/ using document root <none> and password file <none>
I20260926 15:04:44.575356 21032 fs_manager.cc:359] Metadata directory not provided
I20260926 15:04:44.575469 21032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:04:44.575773 21032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 15:04:44 dist-test-slave-7153 krb5kdc[20588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435084, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for HTTP/127.9.123.84@KRBTEST.COM
I20260926 15:04:44.638546 21032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "88445125df9f41b19049010360c4fee1"
format_stamp: "Formatted at 2026-09-26 15:04:44 on dist-test-slave-7153"
server_key: "abdc9aaa3de0eab55e408a8145894e70"
server_key_iv: "9ff57c6603b4c03e9d4cc6a564a0ca01"
server_key_version: "kuduclusterkey@0"
I20260926 15:04:44.639370 21032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "88445125df9f41b19049010360c4fee1"
format_stamp: "Formatted at 2026-09-26 15:04:44 on dist-test-slave-7153"
server_key: "abdc9aaa3de0eab55e408a8145894e70"
server_key_iv: "9ff57c6603b4c03e9d4cc6a564a0ca01"
server_key_version: "kuduclusterkey@0"
I20260926 15:04:44.645231 21032 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260926 15:04:44.662366 21047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:04:44.663969 21032 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 15:04:44.664155 21032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "88445125df9f41b19049010360c4fee1"
format_stamp: "Formatted at 2026-09-26 15:04:44 on dist-test-slave-7153"
server_key: "abdc9aaa3de0eab55e408a8145894e70"
server_key_iv: "9ff57c6603b4c03e9d4cc6a564a0ca01"
server_key_version: "kuduclusterkey@0"
I20260926 15:04:44.664373 21032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:04:44.677155 21032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:04:44.682116 21032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:04:44.682437 21032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:04:44.702266 21032 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:33629
I20260926 15:04:44.702268 21098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:33629 every 8 connection(s)
I20260926 15:04:44.703711 21032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:04:44.705783 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 21032
I20260926 15:04:44.705986 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:04:44.706799 9709 external_mini_cluster.cc:1428] Setting key 81f6b08017cac09f746aa0ab6fa3645a
I20260926 15:04:44.708168 21099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:04:44.725345 21099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88445125df9f41b19049010360c4fee1: Bootstrap starting.
I20260926 15:04:44.729729 21099 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88445125df9f41b19049010360c4fee1: Neither blocks nor log segments found. Creating new log.
I20260926 15:04:44.731099 21099 log.cc:824] T 00000000000000000000000000000000 P 88445125df9f41b19049010360c4fee1: Log is configured to *not* fsync() on all Append() calls
I20260926 15:04:44.734311 21099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88445125df9f41b19049010360c4fee1: No bootstrap required, opened a new log
Sep 26 15:04:44 dist-test-slave-7153 krb5kdc[20588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:04:44.743088 21099 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88445125df9f41b19049010360c4fee1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88445125df9f41b19049010360c4fee1" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 33629 } }
I20260926 15:04:44.743495 21099 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88445125df9f41b19049010360c4fee1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:04:44.743635 21099 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88445125df9f41b19049010360c4fee1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88445125df9f41b19049010360c4fee1, State: Initialized, Role: FOLLOWER
I20260926 15:04:44.744320 21099 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88445125df9f41b19049010360c4fee1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88445125df9f41b19049010360c4fee1" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 33629 } }
I20260926 15:04:44.744587 21099 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88445125df9f41b19049010360c4fee1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:04:44.744778 21099 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88445125df9f41b19049010360c4fee1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:04:44.745023 21099 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88445125df9f41b19049010360c4fee1 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:04:44.747835 21099 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88445125df9f41b19049010360c4fee1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88445125df9f41b19049010360c4fee1" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 33629 } }
I20260926 15:04:44.748350 21099 leader_election.cc:302] T 00000000000000000000000000000000 P 88445125df9f41b19049010360c4fee1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88445125df9f41b19049010360c4fee1; no voters:
I20260926 15:04:44.748993 21099 leader_election.cc:288] T 00000000000000000000000000000000 P 88445125df9f41b19049010360c4fee1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:04:44.749158 21104 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88445125df9f41b19049010360c4fee1 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:04:44.749169 21102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:04:44.709029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37716 (local address 127.9.123.126:33629)
0926 15:04:44.723787 (+ 14758us) server_negotiation.cc:206] Beginning negotiation
0926 15:04:44.723805 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0926 15:04:44.723862 (+ 57us) server_negotiation.cc:407] Connection header received
0926 15:04:44.726647 (+ 2785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:04:44.726681 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:04:44.727230 (+ 549us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:04:44.727766 (+ 536us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:04:44.729473 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:44.731436 (+ 1963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:44.733329 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:44.734329 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:44.734500 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:04:44.739374 (+ 4874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:04:44.739386 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:04:44.739412 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:04:44.739479 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:04:44.744040 (+ 4561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:04:44.744951 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:04:44.744961 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:04:44.744971 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:04:44.745220 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:04:44.745639 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:04:44.745645 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:04:44.745648 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:04:44.746452 (+ 804us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:04:44.746733 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0926 15:04:44.747207 (+ 474us) server_negotiation.cc:299] Negotiation successful
0926 15:04:44.747638 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14492,"thread_start_us":14224,"threads_started":1}
I20260926 15:04:44.756012 21104 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88445125df9f41b19049010360c4fee1 [term 1 LEADER]: Becoming Leader. State: Replica: 88445125df9f41b19049010360c4fee1, State: Running, Role: LEADER
I20260926 15:04:44.756699 21104 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88445125df9f41b19049010360c4fee1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88445125df9f41b19049010360c4fee1" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 33629 } }
I20260926 15:04:44.757630 21099 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88445125df9f41b19049010360c4fee1 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:04:44.759639 21105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88445125df9f41b19049010360c4fee1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88445125df9f41b19049010360c4fee1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88445125df9f41b19049010360c4fee1" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 33629 } } }
I20260926 15:04:44.760047 21105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88445125df9f41b19049010360c4fee1 [sys.catalog]: This master's current role is: LEADER
I20260926 15:04:44.761274 21106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88445125df9f41b19049010360c4fee1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88445125df9f41b19049010360c4fee1. Latest consensus state: current_term: 1 leader_uuid: "88445125df9f41b19049010360c4fee1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88445125df9f41b19049010360c4fee1" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 33629 } } }
I20260926 15:04:44.761564 21106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88445125df9f41b19049010360c4fee1 [sys.catalog]: This master's current role is: LEADER
I20260926 15:04:44.765631 21109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:04:44.772081 21109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:04:44.774398 21099 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 15:04:44.805236 21109 catalog_manager.cc:1347] Generated new cluster ID: ab88594c738f44a6bed46bb41f6b550a
I20260926 15:04:44.805413 21109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:04:44.832597 21109 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:04:44.834784 21109 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:04:44.847627 21109 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88445125df9f41b19049010360c4fee1: Generated new TSK 0
I20260926 15:04:44.848793 21109 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.123.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
I20260926 15:04:45.646538 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.123.84:57103/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:33629
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:38057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 15:04:45 dist-test-slave-7153 krb5kdc[20588](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790435085, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260926 15:04:46.093040 21144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:04:46.093503 21144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:04:46.093713 21144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:04:46.106361 21144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:04:46.106948 21144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:04:46.107197 21144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:04:46.123492 21144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:38057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.123.84:57103/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:33629
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:04:46.125473 21144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:04:46.127242 21144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:04:46.147630 21154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:04:46.149881 21155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:04:46.157341 21157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:04:46.157526 21144 server_base.cc:1034] running on GCE node
I20260926 15:04:46.158493 21144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:04:46.159922 21144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:04:46.161217 21144 hybrid_clock.cc:648] HybridClock initialized: now 1790435086161149 us; error 61 us; skew 500 ppm
I20260926 15:04:46.161598 21144 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:04:46 dist-test-slave-7153 krb5kdc[20588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435086, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:04:46.165020 21144 init.cc:377] Logged in from keytab as kudu/127.9.123.65@KRBTEST.COM (short username kudu)
I20260926 15:04:46.166814 21144 webserver.cc:460] Webserver started at http://127.9.123.65:39299/ using document root <none> and password file <none>
I20260926 15:04:46.167589 21144 fs_manager.cc:359] Metadata directory not provided
I20260926 15:04:46.167704 21144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:04:46.168322 21144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 15:04:46 dist-test-slave-7153 krb5kdc[20588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435086, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for HTTP/127.9.123.84@KRBTEST.COM
I20260926 15:04:46.188530 21144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "46450edaf85d4ce5bad8f4c2e6e15683"
format_stamp: "Formatted at 2026-09-26 15:04:46 on dist-test-slave-7153"
server_key: "86dfee35236a3e0ac81d5da0e1efe2cb"
server_key_iv: "d2b3386153374c87c95cfe75899b9bf7"
server_key_version: "kuduclusterkey@0"
I20260926 15:04:46.190025 21144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "46450edaf85d4ce5bad8f4c2e6e15683"
format_stamp: "Formatted at 2026-09-26 15:04:46 on dist-test-slave-7153"
server_key: "86dfee35236a3e0ac81d5da0e1efe2cb"
server_key_iv: "d2b3386153374c87c95cfe75899b9bf7"
server_key_version: "kuduclusterkey@0"
I20260926 15:04:46.216693 21144 fs_manager.cc:691] Time spent creating directory manager: real 0.026s user 0.007s sys 0.009s
I20260926 15:04:46.256024 21164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:04:46.260141 21144 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.008s sys 0.008s
I20260926 15:04:46.261157 21144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "46450edaf85d4ce5bad8f4c2e6e15683"
format_stamp: "Formatted at 2026-09-26 15:04:46 on dist-test-slave-7153"
server_key: "86dfee35236a3e0ac81d5da0e1efe2cb"
server_key_iv: "d2b3386153374c87c95cfe75899b9bf7"
server_key_version: "kuduclusterkey@0"
I20260926 15:04:46.261585 21144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:04:46.310835 21144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:04:46.315847 21144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:04:46.316150 21144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:04:46.317603 21144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:04:46.319123 21144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:04:46.319202 21144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:04:46.319347 21144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:04:46.319398 21144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 26 15:04:46 dist-test-slave-7153 krb5kdc[20588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435085, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for HTTP/127.9.123.84@KRBTEST.COM
I20260926 15:04:46.483421 21144 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:39011
I20260926 15:04:46.483445 21276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:39011 every 8 connection(s)
I20260926 15:04:46.484887 21144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:04:46.495301 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 21144
I20260926 15:04:46.495507 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:04:46.496235 9709 external_mini_cluster.cc:1428] Setting key acf5c41f09401420e237778acbc5c8e1
Sep 26 15:04:46 dist-test-slave-7153 krb5kdc[20588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435086, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.123.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.66@KRBTEST.COM" created.
I20260926 15:04:46.531924 21280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:04:46.490721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:33629 (local address 127.9.123.65:41379)
0926 15:04:46.491667 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0926 15:04:46.491724 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 15:04:46.494103 (+ 2379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:04:46.510588 (+ 16485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:04:46.510603 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:04:46.511320 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:04:46.512222 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:04:46.512254 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:04:46.514140 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:04:46.514147 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:04:46.516229 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:04:46.516249 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:04:46.517297 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:04:46.517304 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:04:46.517613 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:04:46.518409 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:04:46.518449 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:04:46.520986 (+ 2537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:04:46.525011 (+ 4025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:04:46.525023 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:04:46.525043 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:04:46.525913 (+ 870us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:04:46.526511 (+ 598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:04:46.526543 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:04:46.526548 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:04:46.526853 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:04:46.527655 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:04:46.527665 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:04:46.528142 (+ 477us) client_negotiation.cc:769] Sending connection context
0926 15:04:46.528646 (+ 504us) client_negotiation.cc:240] Negotiation successful
0926 15:04:46.529269 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":360,"threads_started":1}
I20260926 15:04:46.533484 21279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:04:46.489499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:41379 (local address 127.9.123.126:33629)
0926 15:04:46.509740 (+ 20241us) server_negotiation.cc:206] Beginning negotiation
0926 15:04:46.509747 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:04:46.509776 (+ 29us) server_negotiation.cc:407] Connection header received
0926 15:04:46.509850 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:04:46.509855 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:04:46.509948 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:04:46.510113 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:04:46.512597 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:46.513894 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:46.516427 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:46.517011 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:46.517170 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:04:46.521258 (+ 4088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:04:46.521264 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:04:46.521268 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:04:46.521316 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:04:46.524797 (+ 3481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:04:46.526165 (+ 1368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:04:46.526172 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:04:46.526176 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:04:46.526344 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:04:46.527019 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:04:46.527025 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:04:46.527029 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:04:46.527434 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:04:46.527641 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0926 15:04:46.532649 (+ 5008us) server_negotiation.cc:299] Negotiation successful
0926 15:04:46.533090 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20085,"thread_start_us":19938,"threads_started":1}
I20260926 15:04:46.547462 21277 heartbeater.cc:347] Connected to a master server at 127.9.123.126:33629
I20260926 15:04:46.547875 21277 heartbeater.cc:464] Registering TS with master...
I20260926 15:04:46.548874 21277 heartbeater.cc:511] Master 127.9.123.126:33629 requested a full tablet report, sending...
I20260926 15:04:46.554929 21064 ts_manager.cc:194] Registered new tserver with Master: 46450edaf85d4ce5bad8f4c2e6e15683 (127.9.123.65:39011)
I20260926 15:04:46.557209 21064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.65@KRBTEST.COM'} at 127.9.123.65:41379
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
I20260926 15:04:46.571060 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.123.84:57103/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:33629
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:38057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 15:04:46 dist-test-slave-7153 krb5kdc[20588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435085, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for HTTP/127.9.123.84@KRBTEST.COM
W20260926 15:04:47.035522 21283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:04:47.036012 21283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:04:47.036201 21283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:04:47.044222 21283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:04:47.044371 21283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:04:47.044528 21283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:04:47.054908 21283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:38057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.123.84:57103/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:33629
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:04:47.056257 21283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:04:47.057775 21283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:04:47.074250 21297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:04:47.074856 21298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:04:47.074888 21300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:04:47.075556 21283 server_base.cc:1034] running on GCE node
I20260926 15:04:47.076279 21283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:04:47.077589 21283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:04:47.078866 21283 hybrid_clock.cc:648] HybridClock initialized: now 1790435087078793 us; error 66 us; skew 500 ppm
I20260926 15:04:47.079695 21283 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:04:47 dist-test-slave-7153 krb5kdc[20588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435087, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:04:47.083413 21283 init.cc:377] Logged in from keytab as kudu/127.9.123.66@KRBTEST.COM (short username kudu)
I20260926 15:04:47.085150 21283 webserver.cc:460] Webserver started at http://127.9.123.66:45283/ using document root <none> and password file <none>
I20260926 15:04:47.085925 21283 fs_manager.cc:359] Metadata directory not provided
I20260926 15:04:47.086056 21283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:04:47.086452 21283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 15:04:47 dist-test-slave-7153 krb5kdc[20588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435087, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for HTTP/127.9.123.84@KRBTEST.COM
I20260926 15:04:47.105593 21283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "cbfc9bada2f84da5b0818ffbfde7f3c2"
format_stamp: "Formatted at 2026-09-26 15:04:47 on dist-test-slave-7153"
server_key: "487f10968541e7c916ac6e1d3eaef4fc"
server_key_iv: "031ccd5a42c1cbfcad3cf6e90b5f423f"
server_key_version: "kuduclusterkey@0"
I20260926 15:04:47.106745 21283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "cbfc9bada2f84da5b0818ffbfde7f3c2"
format_stamp: "Formatted at 2026-09-26 15:04:47 on dist-test-slave-7153"
server_key: "487f10968541e7c916ac6e1d3eaef4fc"
server_key_iv: "031ccd5a42c1cbfcad3cf6e90b5f423f"
server_key_version: "kuduclusterkey@0"
I20260926 15:04:47.114514 21283 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260926 15:04:47.133522 21307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:04:47.135635 21283 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260926 15:04:47.135867 21283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "cbfc9bada2f84da5b0818ffbfde7f3c2"
format_stamp: "Formatted at 2026-09-26 15:04:47 on dist-test-slave-7153"
server_key: "487f10968541e7c916ac6e1d3eaef4fc"
server_key_iv: "031ccd5a42c1cbfcad3cf6e90b5f423f"
server_key_version: "kuduclusterkey@0"
I20260926 15:04:47.136152 21283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:04:47.168035 21283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:04:47.173631 21283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:04:47.173959 21283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:04:47.175148 21283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:04:47.176996 21283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:04:47.177126 21283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:04:47.177309 21283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:04:47.177376 21283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:04:47.227543 21283 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:40815
I20260926 15:04:47.227584 21419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:40815 every 8 connection(s)
I20260926 15:04:47.229450 21283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:04:47.231487 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 21283
I20260926 15:04:47.231680 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:04:47.232280 9709 external_mini_cluster.cc:1428] Setting key 62553abcaf6bcde33c8644371484ded6
WARNING: no policy specified for kudu/127.9.123.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.67@KRBTEST.COM" created.
Sep 26 15:04:47 dist-test-slave-7153 krb5kdc[20588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435087, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:04:47.283418 21422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:04:47.234544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:58129 (local address 127.9.123.126:33629)
0926 15:04:47.254792 (+ 20248us) server_negotiation.cc:206] Beginning negotiation
0926 15:04:47.254799 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:04:47.256074 (+ 1275us) server_negotiation.cc:407] Connection header received
0926 15:04:47.258160 (+ 2086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:04:47.258166 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:04:47.258278 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:04:47.258350 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:04:47.261248 (+ 2898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.262225 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.263853 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.264453 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.264596 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:04:47.276626 (+ 12030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:04:47.276635 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:04:47.276640 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:04:47.276693 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:04:47.280077 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:04:47.280928 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:04:47.280934 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:04:47.280938 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:04:47.281090 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:04:47.281588 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:04:47.281593 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:04:47.281596 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:04:47.281950 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:04:47.282088 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0926 15:04:47.282833 (+ 745us) server_negotiation.cc:299] Negotiation successful
0926 15:04:47.283059 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20100,"thread_start_us":21535,"threads_started":1}
I20260926 15:04:47.284487 21423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:04:47.235332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:33629 (local address 127.9.123.66:58129)
0926 15:04:47.255425 (+ 20093us) negotiation.cc:107] Waiting for socket to connect
0926 15:04:47.255494 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0926 15:04:47.257800 (+ 2306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:04:47.258925 (+ 1125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:04:47.258938 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:04:47.259549 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:04:47.260962 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:04:47.260995 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.262423 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.262426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:04:47.263689 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:04:47.263701 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.267786 (+ 4085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.267793 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:04:47.268072 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:04:47.268856 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:04:47.268880 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:04:47.271308 (+ 2428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:04:47.280279 (+ 8971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:04:47.280292 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:04:47.280312 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:04:47.280777 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:04:47.281244 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:04:47.281250 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:04:47.281254 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:04:47.281469 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:04:47.282071 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:04:47.282080 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:04:47.282446 (+ 366us) client_negotiation.cc:769] Sending connection context
0926 15:04:47.282686 (+ 240us) client_negotiation.cc:240] Negotiation successful
0926 15:04:47.283226 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19739,"thread_start_us":19509,"threads_started":1}
I20260926 15:04:47.285920 21420 heartbeater.cc:347] Connected to a master server at 127.9.123.126:33629
I20260926 15:04:47.286228 21420 heartbeater.cc:464] Registering TS with master...
I20260926 15:04:47.287539 21420 heartbeater.cc:511] Master 127.9.123.126:33629 requested a full tablet report, sending...
I20260926 15:04:47.289323 21064 ts_manager.cc:194] Registered new tserver with Master: cbfc9bada2f84da5b0818ffbfde7f3c2 (127.9.123.66:40815)
I20260926 15:04:47.290313 21064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.66@KRBTEST.COM'} at 127.9.123.66:58129
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
I20260926 15:04:47.309475 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.123.84:57103/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:33629
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:38057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:04:47.563768 21277 heartbeater.cc:503] Master 127.9.123.126:33629 was elected leader, sending a full tablet report...
W20260926 15:04:47.659794 21426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:04:47.660084 21426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:04:47.660215 21426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:04:47.667574 21426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:04:47.667717 21426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:04:47.667874 21426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:04:47.678516 21426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:38057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.123.84:57103/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:33629
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:04:47.680210 21426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:04:47.681727 21426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:04:47.694739 21431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:04:47.694875 21434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:04:47.694967 21432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:04:47.696336 21426 server_base.cc:1034] running on GCE node
I20260926 15:04:47.696942 21426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:04:47.698031 21426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:04:47.699271 21426 hybrid_clock.cc:648] HybridClock initialized: now 1790435087699208 us; error 50 us; skew 500 ppm
I20260926 15:04:47.699626 21426 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:04:47 dist-test-slave-7153 krb5kdc[20588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435087, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:04:47.703411 21426 init.cc:377] Logged in from keytab as kudu/127.9.123.67@KRBTEST.COM (short username kudu)
I20260926 15:04:47.705331 21426 webserver.cc:460] Webserver started at http://127.9.123.67:37555/ using document root <none> and password file <none>
I20260926 15:04:47.706243 21426 fs_manager.cc:359] Metadata directory not provided
I20260926 15:04:47.706373 21426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:04:47.706709 21426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 15:04:47 dist-test-slave-7153 krb5kdc[20588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435087, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for HTTP/127.9.123.84@KRBTEST.COM
I20260926 15:04:47.724012 21426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "66b1fb1b7939473eb625e69b0c3fc284"
format_stamp: "Formatted at 2026-09-26 15:04:47 on dist-test-slave-7153"
server_key: "fad0348e5155f9be824e01153b737ebf"
server_key_iv: "c0ba23c23f06b1c6282b615301cc95e9"
server_key_version: "kuduclusterkey@0"
I20260926 15:04:47.725236 21426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "66b1fb1b7939473eb625e69b0c3fc284"
format_stamp: "Formatted at 2026-09-26 15:04:47 on dist-test-slave-7153"
server_key: "fad0348e5155f9be824e01153b737ebf"
server_key_iv: "c0ba23c23f06b1c6282b615301cc95e9"
server_key_version: "kuduclusterkey@0"
I20260926 15:04:47.733080 21426 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260926 15:04:47.750470 21441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:04:47.752205 21426 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 15:04:47.752416 21426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "66b1fb1b7939473eb625e69b0c3fc284"
format_stamp: "Formatted at 2026-09-26 15:04:47 on dist-test-slave-7153"
server_key: "fad0348e5155f9be824e01153b737ebf"
server_key_iv: "c0ba23c23f06b1c6282b615301cc95e9"
server_key_version: "kuduclusterkey@0"
I20260926 15:04:47.752633 21426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:04:47.777945 21426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:04:47.782231 21426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:04:47.782539 21426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:04:47.783968 21426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:04:47.785457 21426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:04:47.785549 21426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:04:47.785681 21426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:04:47.785730 21426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:04:47.819617 21426 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:45857
I20260926 15:04:47.819655 21553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:45857 every 8 connection(s)
I20260926 15:04:47.820871 21426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:04:47.822263 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 21426
I20260926 15:04:47.822409 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:04:47.822893 9709 external_mini_cluster.cc:1428] Setting key d0fa1ea47b7fd394a8642b3f11595495
Sep 26 15:04:47 dist-test-slave-7153 krb5kdc[20588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435087, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:04:47.842478 21556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:04:47.823496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:47421 (local address 127.9.123.126:33629)
0926 15:04:47.826305 (+ 2809us) server_negotiation.cc:206] Beginning negotiation
0926 15:04:47.826311 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:04:47.826326 (+ 15us) server_negotiation.cc:407] Connection header received
0926 15:04:47.827656 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:04:47.827661 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:04:47.827752 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:04:47.827877 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:04:47.829539 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.830284 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.831960 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.832410 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.832510 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:04:47.835852 (+ 3342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:04:47.835858 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:04:47.835861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:04:47.835896 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:04:47.839365 (+ 3469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:04:47.840162 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:04:47.840167 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:04:47.840169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:04:47.840305 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:04:47.840774 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:04:47.840777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:04:47.840779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:04:47.841091 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:04:47.841220 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 15:04:47.841989 (+ 769us) server_negotiation.cc:299] Negotiation successful
0926 15:04:47.842166 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2716,"thread_start_us":2144,"threads_started":1}
I20260926 15:04:47.843572 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:04:47.824170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:33629 (local address 127.9.123.67:47421)
0926 15:04:47.825764 (+ 1594us) negotiation.cc:107] Waiting for socket to connect
0926 15:04:47.825821 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 15:04:47.827394 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:04:47.828137 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:04:47.828146 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:04:47.828612 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:04:47.829342 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:04:47.829365 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.830469 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.830473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:04:47.831757 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:04:47.831817 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.832616 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.832622 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:04:47.832874 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:04:47.833523 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:04:47.833550 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:04:47.835571 (+ 2021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:04:47.839505 (+ 3934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:04:47.839513 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:04:47.839529 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:04:47.840025 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:04:47.840435 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:04:47.840439 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:04:47.840442 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:04:47.840667 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:04:47.841230 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:04:47.841237 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:04:47.841603 (+ 366us) client_negotiation.cc:769] Sending connection context
0926 15:04:47.841857 (+ 254us) client_negotiation.cc:240] Negotiation successful
0926 15:04:47.842288 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1304,"thread_start_us":360,"threads_started":1}
I20260926 15:04:47.845032 21554 heartbeater.cc:347] Connected to a master server at 127.9.123.126:33629
I20260926 15:04:47.845361 21554 heartbeater.cc:464] Registering TS with master...
I20260926 15:04:47.846064 21554 heartbeater.cc:511] Master 127.9.123.126:33629 requested a full tablet report, sending...
I20260926 15:04:47.847472 21064 ts_manager.cc:194] Registered new tserver with Master: 66b1fb1b7939473eb625e69b0c3fc284 (127.9.123.67:45857)
I20260926 15:04:47.848242 21064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.67@KRBTEST.COM'} at 127.9.123.67:47421
I20260926 15:04:47.851264 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 15:04:47.869453 21556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:04:47.856292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37720 (local address 127.9.123.126:33629)
0926 15:04:47.856456 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 15:04:47.856461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:04:47.856796 (+ 335us) server_negotiation.cc:407] Connection header received
0926 15:04:47.857032 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:04:47.857036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:04:47.857159 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:04:47.857279 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:04:47.858846 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.859627 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.861031 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.861520 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.861635 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:04:47.863920 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:04:47.863925 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:04:47.863928 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:04:47.863962 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:04:47.866428 (+ 2466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:04:47.867330 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:04:47.867336 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:04:47.867340 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:04:47.867564 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:04:47.868009 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:04:47.868012 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:04:47.868014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:04:47.868292 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:04:47.868500 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0926 15:04:47.868930 (+ 430us) server_negotiation.cc:299] Negotiation successful
0926 15:04:47.869167 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 15:04:47.873799 21064 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:04:47.881384 21064 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:04:47.907483 21567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:04:47.894745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:40815 (local address 127.0.0.1:42076)
0926 15:04:47.896958 (+ 2213us) negotiation.cc:107] Waiting for socket to connect
0926 15:04:47.896974 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:04:47.897124 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:04:47.898173 (+ 1049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:04:47.898181 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:04:47.898208 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:04:47.899212 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:04:47.899229 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.901363 (+ 2134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.901369 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:04:47.903942 (+ 2573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:04:47.903961 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.906382 (+ 2421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.906388 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:04:47.906588 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:04:47.906648 (+ 60us) client_negotiation.cc:769] Sending connection context
0926 15:04:47.906797 (+ 149us) client_negotiation.cc:240] Negotiation successful
0926 15:04:47.907036 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":475,"threads_started":1}
I20260926 15:04:47.908659 21569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:04:47.895533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42076 (local address 127.9.123.66:40815)
0926 15:04:47.896968 (+ 1435us) server_negotiation.cc:206] Beginning negotiation
0926 15:04:47.896977 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:04:47.897239 (+ 262us) server_negotiation.cc:407] Connection header received
0926 15:04:47.897385 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:04:47.897394 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:04:47.897692 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:04:47.897894 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:04:47.899456 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.901174 (+ 1718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.904168 (+ 2994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.905880 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.906010 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:04:47.906230 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0926 15:04:47.908083 (+ 1853us) server_negotiation.cc:299] Negotiation successful
0926 15:04:47.908293 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1294,"thread_start_us":1137,"threads_started":1}
I20260926 15:04:47.913848 21355 tablet_service.cc:1467] Processing CreateTablet for tablet b0136c996ccd43cda0247bf1aa87f189 (DEFAULT_TABLE table=test-table [id=4093902757c54173ba85ba7864c97646]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:04:47.915361 21355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0136c996ccd43cda0247bf1aa87f189. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:04:47.916697 21567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:04:47.896250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:39011 (local address 127.0.0.1:60798)
0926 15:04:47.907761 (+ 11511us) negotiation.cc:107] Waiting for socket to connect
0926 15:04:47.907783 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 15:04:47.907898 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:04:47.910360 (+ 2462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:04:47.910365 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:04:47.910391 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:04:47.910667 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:04:47.910679 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.912702 (+ 2023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.912708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:04:47.914544 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:04:47.914565 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.915849 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.915854 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:04:47.916050 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:04:47.916082 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 15:04:47.916189 (+ 107us) client_negotiation.cc:240] Negotiation successful
0926 15:04:47.916348 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11404,"mutex_wait_us":60,"thread_start_us":12857,"threads_started":1}
I20260926 15:04:47.916850 21568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:04:47.895039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:45857 (local address 127.0.0.1:44472)
0926 15:04:47.896147 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0926 15:04:47.896199 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 15:04:47.896391 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:04:47.910066 (+ 13675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:04:47.910072 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:04:47.910098 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:04:47.910540 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:04:47.910557 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.912199 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.912203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:04:47.914245 (+ 2042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:04:47.914264 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.916070 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.916076 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:04:47.916258 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:04:47.916304 (+ 46us) client_negotiation.cc:769] Sending connection context
0926 15:04:47.916370 (+ 66us) client_negotiation.cc:240] Negotiation successful
0926 15:04:47.916504 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":896,"mutex_wait_us":2,"thread_start_us":707,"threads_started":1}
I20260926 15:04:47.917781 21570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:04:47.895848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44472 (local address 127.9.123.67:45857)
0926 15:04:47.909430 (+ 13582us) server_negotiation.cc:206] Beginning negotiation
0926 15:04:47.909439 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:04:47.909473 (+ 34us) server_negotiation.cc:407] Connection header received
0926 15:04:47.909582 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:04:47.909590 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:04:47.909853 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:04:47.910062 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:04:47.910800 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.912065 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.914420 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.915828 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.916976 (+ 1148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:04:47.917234 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0926 15:04:47.917320 (+ 86us) server_negotiation.cc:299] Negotiation successful
0926 15:04:47.917474 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13412,"thread_start_us":13158,"threads_started":1}
I20260926 15:04:47.918010 21572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:04:47.896862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60798 (local address 127.9.123.65:39011)
0926 15:04:47.909799 (+ 12937us) server_negotiation.cc:206] Beginning negotiation
0926 15:04:47.909805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:04:47.909827 (+ 22us) server_negotiation.cc:407] Connection header received
0926 15:04:47.909905 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:04:47.909911 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:04:47.910119 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:04:47.910835 (+ 716us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:04:47.911060 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.912540 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.914702 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.915691 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.915817 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:04:47.916057 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0926 15:04:47.916250 (+ 193us) server_negotiation.cc:299] Negotiation successful
0926 15:04:47.916411 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12792,"thread_start_us":12647,"threads_started":1}
I20260926 15:04:47.921578 21489 tablet_service.cc:1467] Processing CreateTablet for tablet b0136c996ccd43cda0247bf1aa87f189 (DEFAULT_TABLE table=test-table [id=4093902757c54173ba85ba7864c97646]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:04:47.922343 21212 tablet_service.cc:1467] Processing CreateTablet for tablet b0136c996ccd43cda0247bf1aa87f189 (DEFAULT_TABLE table=test-table [id=4093902757c54173ba85ba7864c97646]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:04:47.923213 21489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0136c996ccd43cda0247bf1aa87f189. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:04:47.923786 21212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0136c996ccd43cda0247bf1aa87f189. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:04:47.928603 21573 tablet_bootstrap.cc:492] T b0136c996ccd43cda0247bf1aa87f189 P cbfc9bada2f84da5b0818ffbfde7f3c2: Bootstrap starting.
I20260926 15:04:47.933318 21573 tablet_bootstrap.cc:654] T b0136c996ccd43cda0247bf1aa87f189 P cbfc9bada2f84da5b0818ffbfde7f3c2: Neither blocks nor log segments found. Creating new log.
I20260926 15:04:47.935276 21573 log.cc:824] T b0136c996ccd43cda0247bf1aa87f189 P cbfc9bada2f84da5b0818ffbfde7f3c2: Log is configured to *not* fsync() on all Append() calls
I20260926 15:04:47.938396 21573 tablet_bootstrap.cc:492] T b0136c996ccd43cda0247bf1aa87f189 P cbfc9bada2f84da5b0818ffbfde7f3c2: No bootstrap required, opened a new log
I20260926 15:04:47.938726 21573 ts_tablet_manager.cc:1397] T b0136c996ccd43cda0247bf1aa87f189 P cbfc9bada2f84da5b0818ffbfde7f3c2: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.006s
I20260926 15:04:47.939383 21576 tablet_bootstrap.cc:492] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284: Bootstrap starting.
I20260926 15:04:47.941443 21575 tablet_bootstrap.cc:492] T b0136c996ccd43cda0247bf1aa87f189 P 46450edaf85d4ce5bad8f4c2e6e15683: Bootstrap starting.
I20260926 15:04:47.944595 21576 tablet_bootstrap.cc:654] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284: Neither blocks nor log segments found. Creating new log.
I20260926 15:04:47.945556 21575 tablet_bootstrap.cc:654] T b0136c996ccd43cda0247bf1aa87f189 P 46450edaf85d4ce5bad8f4c2e6e15683: Neither blocks nor log segments found. Creating new log.
I20260926 15:04:47.945966 21576 log.cc:824] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284: Log is configured to *not* fsync() on all Append() calls
I20260926 15:04:47.946884 21575 log.cc:824] T b0136c996ccd43cda0247bf1aa87f189 P 46450edaf85d4ce5bad8f4c2e6e15683: Log is configured to *not* fsync() on all Append() calls
I20260926 15:04:47.949090 21576 tablet_bootstrap.cc:492] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284: No bootstrap required, opened a new log
I20260926 15:04:47.949433 21576 ts_tablet_manager.cc:1397] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260926 15:04:47.950189 21575 tablet_bootstrap.cc:492] T b0136c996ccd43cda0247bf1aa87f189 P 46450edaf85d4ce5bad8f4c2e6e15683: No bootstrap required, opened a new log
I20260926 15:04:47.950101 21573 raft_consensus.cc:348] T b0136c996ccd43cda0247bf1aa87f189 P cbfc9bada2f84da5b0818ffbfde7f3c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfc9bada2f84da5b0818ffbfde7f3c2" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 40815 } } peers { permanent_uuid: "66b1fb1b7939473eb625e69b0c3fc284" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 45857 } } peers { permanent_uuid: "46450edaf85d4ce5bad8f4c2e6e15683" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 39011 } }
I20260926 15:04:47.950481 21575 ts_tablet_manager.cc:1397] T b0136c996ccd43cda0247bf1aa87f189 P 46450edaf85d4ce5bad8f4c2e6e15683: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260926 15:04:47.950600 21573 raft_consensus.cc:374] T b0136c996ccd43cda0247bf1aa87f189 P cbfc9bada2f84da5b0818ffbfde7f3c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:04:47.950699 21573 raft_consensus.cc:729] T b0136c996ccd43cda0247bf1aa87f189 P cbfc9bada2f84da5b0818ffbfde7f3c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbfc9bada2f84da5b0818ffbfde7f3c2, State: Initialized, Role: FOLLOWER
I20260926 15:04:47.951380 21573 consensus_queue.cc:260] T b0136c996ccd43cda0247bf1aa87f189 P cbfc9bada2f84da5b0818ffbfde7f3c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfc9bada2f84da5b0818ffbfde7f3c2" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 40815 } } peers { permanent_uuid: "66b1fb1b7939473eb625e69b0c3fc284" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 45857 } } peers { permanent_uuid: "46450edaf85d4ce5bad8f4c2e6e15683" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 39011 } }
I20260926 15:04:47.952577 21420 heartbeater.cc:503] Master 127.9.123.126:33629 was elected leader, sending a full tablet report...
I20260926 15:04:47.953274 21573 ts_tablet_manager.cc:1428] T b0136c996ccd43cda0247bf1aa87f189 P cbfc9bada2f84da5b0818ffbfde7f3c2: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260926 15:04:47.960529 21575 raft_consensus.cc:348] T b0136c996ccd43cda0247bf1aa87f189 P 46450edaf85d4ce5bad8f4c2e6e15683 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfc9bada2f84da5b0818ffbfde7f3c2" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 40815 } } peers { permanent_uuid: "66b1fb1b7939473eb625e69b0c3fc284" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 45857 } } peers { permanent_uuid: "46450edaf85d4ce5bad8f4c2e6e15683" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 39011 } }
I20260926 15:04:47.960947 21575 raft_consensus.cc:374] T b0136c996ccd43cda0247bf1aa87f189 P 46450edaf85d4ce5bad8f4c2e6e15683 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:04:47.961041 21575 raft_consensus.cc:729] T b0136c996ccd43cda0247bf1aa87f189 P 46450edaf85d4ce5bad8f4c2e6e15683 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46450edaf85d4ce5bad8f4c2e6e15683, State: Initialized, Role: FOLLOWER
I20260926 15:04:47.961170 21576 raft_consensus.cc:348] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfc9bada2f84da5b0818ffbfde7f3c2" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 40815 } } peers { permanent_uuid: "66b1fb1b7939473eb625e69b0c3fc284" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 45857 } } peers { permanent_uuid: "46450edaf85d4ce5bad8f4c2e6e15683" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 39011 } }
I20260926 15:04:47.961618 21576 raft_consensus.cc:374] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:04:47.961723 21576 raft_consensus.cc:729] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66b1fb1b7939473eb625e69b0c3fc284, State: Initialized, Role: FOLLOWER
I20260926 15:04:47.961678 21575 consensus_queue.cc:260] T b0136c996ccd43cda0247bf1aa87f189 P 46450edaf85d4ce5bad8f4c2e6e15683 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfc9bada2f84da5b0818ffbfde7f3c2" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 40815 } } peers { permanent_uuid: "66b1fb1b7939473eb625e69b0c3fc284" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 45857 } } peers { permanent_uuid: "46450edaf85d4ce5bad8f4c2e6e15683" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 39011 } }
I20260926 15:04:47.962306 21576 consensus_queue.cc:260] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfc9bada2f84da5b0818ffbfde7f3c2" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 40815 } } peers { permanent_uuid: "66b1fb1b7939473eb625e69b0c3fc284" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 45857 } } peers { permanent_uuid: "46450edaf85d4ce5bad8f4c2e6e15683" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 39011 } }
I20260926 15:04:47.963359 21554 heartbeater.cc:503] Master 127.9.123.126:33629 was elected leader, sending a full tablet report...
I20260926 15:04:47.963574 21575 ts_tablet_manager.cc:1428] T b0136c996ccd43cda0247bf1aa87f189 P 46450edaf85d4ce5bad8f4c2e6e15683: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260926 15:04:47.965194 21576 ts_tablet_manager.cc:1428] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284: Time spent starting tablet: real 0.016s user 0.013s sys 0.000s
I20260926 15:04:47.967653 21581 raft_consensus.cc:482] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:04:47.967955 21581 raft_consensus.cc:504] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfc9bada2f84da5b0818ffbfde7f3c2" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 40815 } } peers { permanent_uuid: "66b1fb1b7939473eb625e69b0c3fc284" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 45857 } } peers { permanent_uuid: "46450edaf85d4ce5bad8f4c2e6e15683" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 39011 } }
I20260926 15:04:47.969918 21581 leader_election.cc:288] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cbfc9bada2f84da5b0818ffbfde7f3c2 (127.9.123.66:40815), 46450edaf85d4ce5bad8f4c2e6e15683 (127.9.123.65:39011)
I20260926 15:04:47.976936 21569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:04:47.970301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:48799 (local address 127.9.123.66:40815)
0926 15:04:47.970505 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 15:04:47.970511 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:04:47.970533 (+ 22us) server_negotiation.cc:407] Connection header received
0926 15:04:47.970598 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:04:47.970603 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:04:47.970658 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:04:47.970727 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:04:47.971313 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.972311 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.975008 (+ 2697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.976047 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.976196 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:04:47.976274 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 15:04:47.976534 (+ 260us) server_negotiation.cc:299] Negotiation successful
0926 15:04:47.976664 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260926 15:04:47.976941 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:04:47.970142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:40815 (local address 127.9.123.67:48799)
0926 15:04:47.970344 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0926 15:04:47.970364 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 15:04:47.970481 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:04:47.970955 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:04:47.970960 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:04:47.970992 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:04:47.971163 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:04:47.971171 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.972435 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.972440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:04:47.974752 (+ 2312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:04:47.974771 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.976192 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.976198 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:04:47.976379 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:04:47.976404 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 15:04:47.976493 (+ 89us) client_negotiation.cc:240] Negotiation successful
0926 15:04:47.976649 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":45}
I20260926 15:04:47.977174 21572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:04:47.970696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:44937 (local address 127.9.123.65:39011)
0926 15:04:47.970961 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0926 15:04:47.970967 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:04:47.971248 (+ 281us) server_negotiation.cc:407] Connection header received
0926 15:04:47.971389 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:04:47.971393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:04:47.971465 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:04:47.971528 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:04:47.972078 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.973078 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.975306 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.976265 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.976418 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:04:47.976493 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 15:04:47.976760 (+ 267us) server_negotiation.cc:299] Negotiation successful
0926 15:04:47.976905 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260926 15:04:47.977293 21582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:04:47.970517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:39011 (local address 127.9.123.67:44937)
0926 15:04:47.971155 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0926 15:04:47.971170 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 15:04:47.971280 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:04:47.971581 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:04:47.971585 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:04:47.971611 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:04:47.971942 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:04:47.971954 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.973483 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.973487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:04:47.975139 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:04:47.975151 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:04:47.976466 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:04:47.976472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:04:47.976615 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:04:47.976637 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 15:04:47.976721 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 15:04:47.976854 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"mutex_wait_us":20,"thread_start_us":376,"threads_started":1}
I20260926 15:04:47.978050 21375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0136c996ccd43cda0247bf1aa87f189" candidate_uuid: "66b1fb1b7939473eb625e69b0c3fc284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbfc9bada2f84da5b0818ffbfde7f3c2" is_pre_election: true
I20260926 15:04:47.978209 21232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0136c996ccd43cda0247bf1aa87f189" candidate_uuid: "66b1fb1b7939473eb625e69b0c3fc284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46450edaf85d4ce5bad8f4c2e6e15683" is_pre_election: true
I20260926 15:04:47.978572 21375 raft_consensus.cc:2454] T b0136c996ccd43cda0247bf1aa87f189 P cbfc9bada2f84da5b0818ffbfde7f3c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66b1fb1b7939473eb625e69b0c3fc284 in term 0.
I20260926 15:04:47.978747 21232 raft_consensus.cc:2454] T b0136c996ccd43cda0247bf1aa87f189 P 46450edaf85d4ce5bad8f4c2e6e15683 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66b1fb1b7939473eb625e69b0c3fc284 in term 0.
I20260926 15:04:47.979396 21442 leader_election.cc:302] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66b1fb1b7939473eb625e69b0c3fc284, cbfc9bada2f84da5b0818ffbfde7f3c2; no voters:
I20260926 15:04:47.979764 21581 raft_consensus.cc:2781] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:04:47.979930 21581 raft_consensus.cc:482] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:04:47.980036 21581 raft_consensus.cc:3037] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:04:47.982273 21581 raft_consensus.cc:504] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfc9bada2f84da5b0818ffbfde7f3c2" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 40815 } } peers { permanent_uuid: "66b1fb1b7939473eb625e69b0c3fc284" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 45857 } } peers { permanent_uuid: "46450edaf85d4ce5bad8f4c2e6e15683" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 39011 } }
I20260926 15:04:47.982975 21581 leader_election.cc:288] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284 [CANDIDATE]: Term 1 election: Requested vote from peers cbfc9bada2f84da5b0818ffbfde7f3c2 (127.9.123.66:40815), 46450edaf85d4ce5bad8f4c2e6e15683 (127.9.123.65:39011)
I20260926 15:04:47.983491 21375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0136c996ccd43cda0247bf1aa87f189" candidate_uuid: "66b1fb1b7939473eb625e69b0c3fc284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbfc9bada2f84da5b0818ffbfde7f3c2"
I20260926 15:04:47.983672 21375 raft_consensus.cc:3037] T b0136c996ccd43cda0247bf1aa87f189 P cbfc9bada2f84da5b0818ffbfde7f3c2 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:04:47.983608 21232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0136c996ccd43cda0247bf1aa87f189" candidate_uuid: "66b1fb1b7939473eb625e69b0c3fc284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46450edaf85d4ce5bad8f4c2e6e15683"
I20260926 15:04:47.983861 21232 raft_consensus.cc:3037] T b0136c996ccd43cda0247bf1aa87f189 P 46450edaf85d4ce5bad8f4c2e6e15683 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:04:47.986037 21375 raft_consensus.cc:2454] T b0136c996ccd43cda0247bf1aa87f189 P cbfc9bada2f84da5b0818ffbfde7f3c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66b1fb1b7939473eb625e69b0c3fc284 in term 1.
I20260926 15:04:47.986454 21442 leader_election.cc:302] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66b1fb1b7939473eb625e69b0c3fc284, cbfc9bada2f84da5b0818ffbfde7f3c2; no voters:
I20260926 15:04:47.986610 21232 raft_consensus.cc:2454] T b0136c996ccd43cda0247bf1aa87f189 P 46450edaf85d4ce5bad8f4c2e6e15683 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66b1fb1b7939473eb625e69b0c3fc284 in term 1.
I20260926 15:04:47.986728 21581 raft_consensus.cc:2781] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:04:47.987216 21581 raft_consensus.cc:686] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284 [term 1 LEADER]: Becoming Leader. State: Replica: 66b1fb1b7939473eb625e69b0c3fc284, State: Running, Role: LEADER
I20260926 15:04:47.987699 21581 consensus_queue.cc:237] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfc9bada2f84da5b0818ffbfde7f3c2" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 40815 } } peers { permanent_uuid: "66b1fb1b7939473eb625e69b0c3fc284" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 45857 } } peers { permanent_uuid: "46450edaf85d4ce5bad8f4c2e6e15683" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 39011 } }
I20260926 15:04:47.992795 21064 catalog_manager.cc:5495] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66b1fb1b7939473eb625e69b0c3fc284 (127.9.123.67). New cstate: current_term: 1 leader_uuid: "66b1fb1b7939473eb625e69b0c3fc284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbfc9bada2f84da5b0818ffbfde7f3c2" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 40815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66b1fb1b7939473eb625e69b0c3fc284" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 45857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46450edaf85d4ce5bad8f4c2e6e15683" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 39011 } health_report { overall_health: UNKNOWN } } }
W20260926 15:04:48.002574 21421 tablet.cc:2369] T b0136c996ccd43cda0247bf1aa87f189 P cbfc9bada2f84da5b0818ffbfde7f3c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 15:04:48.012378 21278 tablet.cc:2369] T b0136c996ccd43cda0247bf1aa87f189 P 46450edaf85d4ce5bad8f4c2e6e15683: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:04:48.031744 21570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:04:48.026796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44476 (local address 127.9.123.67:45857)
0926 15:04:48.026964 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 15:04:48.026969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:04:48.026986 (+ 17us) server_negotiation.cc:407] Connection header received
0926 15:04:48.027074 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:04:48.027077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:04:48.027153 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:04:48.027284 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:04:48.027568 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:48.028251 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:48.029707 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:48.030115 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:48.030217 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:04:48.030710 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:04:48.031233 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:04:48.031360 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0926 15:04:48.031468 (+ 108us) server_negotiation.cc:299] Negotiation successful
0926 15:04:48.031540 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260926 15:04:48.040642 21232 raft_consensus.cc:1261] T b0136c996ccd43cda0247bf1aa87f189 P 46450edaf85d4ce5bad8f4c2e6e15683 [term 1 FOLLOWER]: Refusing update from remote peer 66b1fb1b7939473eb625e69b0c3fc284: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:04:48.040644 21375 raft_consensus.cc:1261] T b0136c996ccd43cda0247bf1aa87f189 P cbfc9bada2f84da5b0818ffbfde7f3c2 [term 1 FOLLOWER]: Refusing update from remote peer 66b1fb1b7939473eb625e69b0c3fc284: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:04:48.041610 21583 consensus_queue.cc:1035] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46450edaf85d4ce5bad8f4c2e6e15683" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 39011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:04:48.041973 21581 consensus_queue.cc:1035] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbfc9bada2f84da5b0818ffbfde7f3c2" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 40815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:04:48.051426 21586 mvcc.cc:205] Tried to move back new op lower bound from 7333622120604512256 to 7333622120401481728. Current Snapshot: MvccSnapshot[applied={T|T < 7333622120604512256}]
I20260926 15:04:48.052990 21587 mvcc.cc:205] Tried to move back new op lower bound from 7333622120604512256 to 7333622120401481728. Current Snapshot: MvccSnapshot[applied={T|T < 7333622120604512256}]
I20260926 15:04:48.067955 21063 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:04:48.068262 21063 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:04:48.072897 21063 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 88445125df9f41b19049010360c4fee1: Sending DeleteTablet for 3 replicas of tablet b0136c996ccd43cda0247bf1aa87f189
I20260926 15:04:48.074254 21489 tablet_service.cc:1514] Processing DeleteTablet for tablet b0136c996ccd43cda0247bf1aa87f189 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:04:48 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:44472
I20260926 15:04:48.074245 21355 tablet_service.cc:1514] Processing DeleteTablet for tablet b0136c996ccd43cda0247bf1aa87f189 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:04:48 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:42076
I20260926 15:04:48.074512 21212 tablet_service.cc:1514] Processing DeleteTablet for tablet b0136c996ccd43cda0247bf1aa87f189 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:04:48 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:60798
I20260926 15:04:48.076965 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 21144
I20260926 15:04:48.084946 21598 tablet_replica.cc:330] T b0136c996ccd43cda0247bf1aa87f189 P 46450edaf85d4ce5bad8f4c2e6e15683: stopping tablet replica
I20260926 15:04:48.085583 21598 raft_consensus.cc:2229] T b0136c996ccd43cda0247bf1aa87f189 P 46450edaf85d4ce5bad8f4c2e6e15683 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:04:48.086143 21598 raft_consensus.cc:2258] T b0136c996ccd43cda0247bf1aa87f189 P 46450edaf85d4ce5bad8f4c2e6e15683 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:04:48.089185 21596 tablet_replica.cc:330] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284: stopping tablet replica
I20260926 15:04:48.089810 21598 ts_tablet_manager.cc:1905] T b0136c996ccd43cda0247bf1aa87f189 P 46450edaf85d4ce5bad8f4c2e6e15683: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:04:48.093184 21596 raft_consensus.cc:2229] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284 [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:04:48.093952 21596 raft_consensus.cc:2258] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:04:48.096400 21596 ts_tablet_manager.cc:1905] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:04:48.097697 21597 tablet_replica.cc:330] T b0136c996ccd43cda0247bf1aa87f189 P cbfc9bada2f84da5b0818ffbfde7f3c2: stopping tablet replica
I20260926 15:04:48.098203 21597 raft_consensus.cc:2229] T b0136c996ccd43cda0247bf1aa87f189 P cbfc9bada2f84da5b0818ffbfde7f3c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:04:48.098762 21597 raft_consensus.cc:2258] T b0136c996ccd43cda0247bf1aa87f189 P cbfc9bada2f84da5b0818ffbfde7f3c2 [term 1 FOLLOWER]: Raft consensus is shut down!
Sep 26 15:04:48 dist-test-slave-7153 krb5kdc[20588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.65@KRBTEST.COM
I20260926 15:04:48.103691 21598 ts_tablet_manager.cc:1918] T b0136c996ccd43cda0247bf1aa87f189 P 46450edaf85d4ce5bad8f4c2e6e15683: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:04:48.103880 21598 log.cc:1196] T b0136c996ccd43cda0247bf1aa87f189 P 46450edaf85d4ce5bad8f4c2e6e15683: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-0/wal/wals/b0136c996ccd43cda0247bf1aa87f189
I20260926 15:04:48.104401 21598 ts_tablet_manager.cc:1939] T b0136c996ccd43cda0247bf1aa87f189 P 46450edaf85d4ce5bad8f4c2e6e15683: Deleting consensus metadata
I20260926 15:04:48.106483 21048 catalog_manager.cc:4841] TS 46450edaf85d4ce5bad8f4c2e6e15683 (127.9.123.65:39011): tablet b0136c996ccd43cda0247bf1aa87f189 (table test-table [id=4093902757c54173ba85ba7864c97646]) successfully deleted
I20260926 15:04:48.107401 21572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:04:48.078630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60800 (local address 127.9.123.65:39011)
0926 15:04:48.078829 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 15:04:48.078835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:04:48.083547 (+ 4712us) server_negotiation.cc:407] Connection header received
0926 15:04:48.083764 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:04:48.083770 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:04:48.083868 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:04:48.083970 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:04:48.091669 (+ 7699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:48.092949 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:48.095427 (+ 2478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:48.096191 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:48.096306 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:04:48.101262 (+ 4956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:04:48.101281 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:04:48.101299 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:04:48.101375 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:04:48.104349 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:04:48.105150 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:04:48.105160 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:04:48.105163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:04:48.105310 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:04:48.105716 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:04:48.105720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:04:48.105722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:04:48.106084 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:04:48.106372 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0926 15:04:48.106939 (+ 567us) server_negotiation.cc:299] Negotiation successful
0926 15:04:48.107132 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 15:04:48.108633 21272 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:04:48.111254 21597 ts_tablet_manager.cc:1905] T b0136c996ccd43cda0247bf1aa87f189 P cbfc9bada2f84da5b0818ffbfde7f3c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:04:48.120869 21597 ts_tablet_manager.cc:1918] T b0136c996ccd43cda0247bf1aa87f189 P cbfc9bada2f84da5b0818ffbfde7f3c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:04:48.120878 21596 ts_tablet_manager.cc:1918] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:04:48.121059 21597 log.cc:1196] T b0136c996ccd43cda0247bf1aa87f189 P cbfc9bada2f84da5b0818ffbfde7f3c2: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/b0136c996ccd43cda0247bf1aa87f189
I20260926 15:04:48.121063 21596 log.cc:1196] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/b0136c996ccd43cda0247bf1aa87f189
I20260926 15:04:48.121552 21597 ts_tablet_manager.cc:1939] T b0136c996ccd43cda0247bf1aa87f189 P cbfc9bada2f84da5b0818ffbfde7f3c2: Deleting consensus metadata
I20260926 15:04:48.121584 21596 ts_tablet_manager.cc:1939] T b0136c996ccd43cda0247bf1aa87f189 P 66b1fb1b7939473eb625e69b0c3fc284: Deleting consensus metadata
I20260926 15:04:48.123320 21048 catalog_manager.cc:4841] TS cbfc9bada2f84da5b0818ffbfde7f3c2 (127.9.123.66:40815): tablet b0136c996ccd43cda0247bf1aa87f189 (table test-table [id=4093902757c54173ba85ba7864c97646]) successfully deleted
I20260926 15:04:48.123756 21049 catalog_manager.cc:4841] TS 66b1fb1b7939473eb625e69b0c3fc284 (127.9.123.67:45857): tablet b0136c996ccd43cda0247bf1aa87f189 (table test-table [id=4093902757c54173ba85ba7864c97646]) successfully deleted
I20260926 15:04:48.207374 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 21144
I20260926 15:04:48.221944 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 21283
Sep 26 15:04:48 dist-test-slave-7153 krb5kdc[20588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.66@KRBTEST.COM
I20260926 15:04:48.237634 21569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:04:48.222743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42082 (local address 127.9.123.66:40815)
0926 15:04:48.222911 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 15:04:48.222917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:04:48.222937 (+ 20us) server_negotiation.cc:407] Connection header received
0926 15:04:48.223055 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:04:48.223061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:04:48.223129 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:04:48.223228 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:04:48.224445 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:48.225239 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:48.226515 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:48.226966 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:48.227092 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:04:48.231370 (+ 4278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:04:48.231384 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:04:48.231397 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:04:48.231436 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:04:48.234410 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:04:48.235150 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:04:48.235158 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:04:48.235161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:04:48.235304 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:04:48.235723 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:04:48.235727 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:04:48.235729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:04:48.236186 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:04:48.236487 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0926 15:04:48.236870 (+ 383us) server_negotiation.cc:299] Negotiation successful
0926 15:04:48.237306 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260926 15:04:48.238315 21415 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:04:48.328598 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 21283
I20260926 15:04:48.347476 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 21426
Sep 26 15:04:48 dist-test-slave-7153 krb5kdc[20588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.67@KRBTEST.COM
I20260926 15:04:48.364006 21570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:04:48.348218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44488 (local address 127.9.123.67:45857)
0926 15:04:48.348386 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 15:04:48.348391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:04:48.348548 (+ 157us) server_negotiation.cc:407] Connection header received
0926 15:04:48.348694 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:04:48.348700 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:04:48.348793 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:04:48.348913 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:04:48.350148 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:48.351307 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:48.352571 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:04:48.353410 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:04:48.353551 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:04:48.358143 (+ 4592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:04:48.358159 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:04:48.358175 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:04:48.358219 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:04:48.361091 (+ 2872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:04:48.361847 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:04:48.361857 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:04:48.361860 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:04:48.362054 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:04:48.362404 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:04:48.362409 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:04:48.362412 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:04:48.362866 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:04:48.363189 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0926 15:04:48.363347 (+ 158us) server_negotiation.cc:299] Negotiation successful
0926 15:04:48.363613 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260926 15:04:48.364734 21549 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:04:48.484670 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 21426
I20260926 15:04:48.499938 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 21032
I20260926 15:04:48.501277 21094 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:04:48.612025 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 21032
I20260926 15:04:48.631799 9709 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 15:04:48.967443 9709 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 15:04:48.967664 9709 mini_ranger.cc:66] Stopping Ranger...
I20260926 15:04:49.313714 9709 mini_ranger.cc:68] Stopped Ranger
2026-09-26 15:04:49.314 UTC [20618] LOG: received smart shutdown request
2026-09-26 15:04:49.318 UTC [20618] LOG: background worker "logical replication launcher" (PID 20627) exited with exit code 1
2026-09-26 15:04:49.318 UTC [20622] LOG: shutting down
2026-09-26 15:04:49.494 UTC [20618] LOG: database system is shut down
2026-09-26T15:04:49Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51036 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:04:49 dist-test-slave-7153 krb5kdc[21612](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:04:49 dist-test-slave-7153 krb5kdc[21612](info): set up 2 sockets
Sep 26 15:04:49 dist-test-slave-7153 krb5kdc[21612](info): commencing operation
krb5kdc: starting...
W20260926 15:04:50.641443 9709 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.002s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:04:50 dist-test-slave-7153 krb5kdc[21612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:04:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:04:50Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.123.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.123.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.123.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/rangeradmin_127.9.123.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.123.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/rangeradmin_127.9.123.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.123.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.123.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.123.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/rangerlookup_127.9.123.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.123.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/rangerlookup_127.9.123.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.123.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.123.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.123.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/HTTP_127.9.123.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.123.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/HTTP_127.9.123.84@KRBTEST.COM.keytab.
I20260926 15:04:50.956615 9709 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskR962dp/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/postgres -l logfile start
2026-09-26 15:04:53.763 UTC [21642] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-26 15:04:53.763 UTC [21642] LOG: listening on IPv4 address "127.9.123.84", port 57049
2026-09-26 15:04:53.769 UTC [21642] LOG: listening on Unix socket "/tmp/.s.PGSQL.57049"
2026-09-26 15:04:53.799 UTC [21645] LOG: database system was shut down at 2026-09-26 15:04:52 UTC
2026-09-26 15:04:53.806 UTC [21642] LOG: database system is ready to accept connections
I20260926 15:04:54.705411 9709 mini_postgres.cc:96] Postgres bound to 57049
2026-09-26 15:04:54.723 UTC [21653] FATAL: database "slave" does not exist
127.9.123.84:57049 - accepting connections
I20260926 15:04:54.724720 9709 mini_ranger.cc:161] Starting Ranger...
I20260926 15:04:54.753748 9709 mini_ranger.cc:84] Created miniranger Postgres user
I20260926 15:04:55.317027 9709 mini_ranger.cc:87] Created ranger Postgres database
I20260926 15:04:55.317160 9709 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ranger-admin
2026-09-26 15:04:55,606 [I] DB FLAVOR :POSTGRES
2026-09-26 15:04:55,607 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 15:04:55,607 [I] Checking connection..
2026-09-26 15:04:55,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:57049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-26 15:04:55,851 [I] Checking connection passed.
2026-09-26 15:04:55,852 [I] --------- Verifying version history table ---------
2026-09-26 15:04:55,852 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:57049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 15:04:56,123 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 15:04:56,124 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:57049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 15:04:56,374 [I] Table x_db_version_h does not exist in database ranger
2026-09-26 15:04:56,374 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-26 15:04:56,374 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:57049/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-26 15:04:56.628 UTC [21717] WARNING: there is no transaction in progress
2026-09-26 15:04:56,634 [I] create_dbversion_catalog.sql file imported successfully
2026-09-26 15:04:56,634 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:57049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-26 15:04:56,880 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-26 15:04:56,881 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 15:04:56,881 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:57049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-26 15:04:57,113 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:57049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-26 15:04:57,373 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:57049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7153.c.gcp-upstream.internal','N') ;"
2026-09-26 15:04:57,632 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:57049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 15:04:57,882 [I] Table x_portal_user does not exist in database ranger
2026-09-26 15:04:57,882 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:57049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 15:04:58,149 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-26 15:04:58,149 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:57049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 15:04:58,402 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:57049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 15:04:58,653 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-26 15:04:58,653 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:57049/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-26 15:04:59.710 UTC [21825] WARNING: there is no transaction in progress
2026-09-26 15:04:59.727 UTC [21825] WARNING: there is no transaction in progress
2026-09-26 15:04:59.744 UTC [21825] WARNING: there is no transaction in progress
2026-09-26 15:04:59.760 UTC [21825] WARNING: there is no transaction in progress
2026-09-26 15:04:59.777 UTC [21825] WARNING: there is no transaction in progress
2026-09-26 15:04:59.797 UTC [21825] WARNING: there is no transaction in progress
2026-09-26 15:04:59.856 UTC [21825] WARNING: there is no transaction in progress
2026-09-26 15:04:59.863 UTC [21825] WARNING: there is no transaction in progress
2026-09-26 15:04:59.878 UTC [21825] WARNING: there is no transaction in progress
2026-09-26 15:04:59.890 UTC [21825] WARNING: there is no transaction in progress
2026-09-26 15:04:59.900 UTC [21825] WARNING: there is no transaction in progress
2026-09-26 15:04:59.915 UTC [21825] WARNING: there is no transaction in progress
2026-09-26 15:04:59.925 UTC [21825] WARNING: there is no transaction in progress
2026-09-26 15:04:59.931 UTC [21825] WARNING: there is no transaction in progress
2026-09-26 15:04:59.938 UTC [21825] WARNING: there is no transaction in progress
2026-09-26 15:04:59.945 UTC [21825] WARNING: there is no transaction in progress
2026-09-26 15:05:00,699 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-26 15:05:00,700 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:57049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-26 15:05:00,962 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-26 15:05:00,963 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:57049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-26 15:05:01,218 [I] Table x_portal_user already exists in database 'ranger'
2026-09-26 15:05:01,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:57049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-26 15:05:01,490 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-26 15:05:01,490 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:57049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-26 15:05:01,747 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:57049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-26 15:05:02,008 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:57049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7153.c.gcp-upstream.internal';"
2026-09-26 15:05:02,249 [I] CORE_DB_SCHEMA import status has been updated
2026-09-26 15:05:02,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:57049/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-26 15:05:02,502 [I] DB_PATCHES have already been applied
I20260926 15:05:02.508116 9709 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ranger-admin:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskR962dp/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260926 15:05:02.508215 9709 mini_ranger.cc:193] Using host: 127.9.123.84
I20260926 15:05:02.521749 9709 mini_ranger.cc:227] Ranger admin URL: http://127.9.123.84:58253
Sep 26, 2026 3:05:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 3:05:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 3:05:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ranger-admin/ews/webapp
Sep 26, 2026 3:05:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 26, 2026 3:05:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 26, 2026 3:05:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ranger-admin/ews/webapp].
Sep 26, 2026 3:05:03 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260926 15:05:03.559623 9709 mini_ranger.cc:160] Time spent starting Ranger: real 8.835s user 0.004s sys 0.036s
Sep 26, 2026 3:05:24 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 26, 2026 3:05:24 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 26, 2026 3:05:24 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 26, 2026 3:05:24 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 26, 2026 3:05:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 26, 2026 3:05:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 26, 2026 3:05:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 26, 2026 3:05:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 26, 2026 3:05:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 26, 2026 3:05:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 26, 2026 3:05:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 26, 2026 3:05:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 26, 2026 3:05:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 26, 2026 3:05:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 26, 2026 3:05:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 26, 2026 3:05:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 26, 2026 3:05:24 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 26, 2026 3:05:25 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260926 15:05:26.041857 9709 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.123.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.123.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.123.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/rangerkms_127.9.123.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.123.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/rangerkms_127.9.123.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.123.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.123.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.123.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/HTTP_127.9.123.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.123.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/HTTP_127.9.123.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 26 15:05:26 dist-test-slave-7153 krb5kdc[21612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435126, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260926 15:05:26.227535 9709 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260926 15:05:26.248945 9709 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260926 15:05:27.387799 9709 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260926 15:05:27.387907 9709 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ranger-kms
I20260926 15:05:27.387946 9709 mini_ranger_kms.cc:227] Using postgres at 127.9.123.84:57049
2026-09-26 15:05:27,742 [I] DB FLAVOR :POSTGRES
2026-09-26 15:05:27,743 [I] --------- Verifying Ranger DB connection ---------
2026-09-26 15:05:27,743 [I] Checking connection
2026-09-26 15:05:27,743 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:57049/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-26 15:05:28,001 [I] connection success
2026-09-26 15:05:28,001 [I] --------- Verifying Ranger DB tables ---------
2026-09-26 15:05:28,002 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-26 15:05:28,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:57049/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-26 15:05:28,271 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-26 15:05:28,271 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-26 15:05:28,272 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-26 15:05:28,272 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.123.84:57049/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-26 15:05:28,563 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260926 15:05:29.107899 9709 mini_ranger_kms.cc:310] Created kms service
I20260926 15:05:29.322178 9709 mini_ranger_kms.cc:326] Created kudu user
I20260926 15:05:29.378906 9709 mini_ranger_kms.cc:343] Created rangerkms user
Sep 26 15:05:29 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435126, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.123.84@KRBTEST.COM
I20260926 15:05:29.851871 9709 mini_ranger_kms.cc:384] Added ranger policy
I20260926 15:05:29.852183 9709 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ranger-kms:/tmp/dist-test-taskR962dp/build/asan/bin/postgresql.jar:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskR962dp/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260926 15:05:29.852234 9709 mini_ranger_kms.cc:240] Using host: 127.9.123.84
I20260926 15:05:29.866348 9709 mini_ranger_kms.cc:276] Ranger KMS PID: 22006
I20260926 15:05:29.866483 9709 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.123.84:33061
Sep 26, 2026 3:05:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 26, 2026 3:05:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 26, 2026 3:05:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 26, 2026 3:05:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 26, 2026 3:05:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ranger-kms/ews/webapp].
Sep 26, 2026 3:05:31 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260926 15:05:31.985711 9709 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.758s user 0.068s sys 0.301s
I20260926 15:05:31.985944 9709 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260926 15:05:31.986123 9709 mini_ranger_kms.cc:401] 127.9.123.84:33061/kms/v1/keys
Sep 26 15:05:32 dist-test-slave-7153 krb5kdc[21612](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790435132, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.123.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 26 15:05:32 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435132, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.123.84@KRBTEST.COM for HTTP/127.9.123.84@KRBTEST.COM
Sep 26 15:05:32 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435132, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.123.84@KRBTEST.COM for HTTP/127.9.123.84@KRBTEST.COM
Sep 26 15:05:35 dist-test-slave-7153 krb5kdc[21612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:05:35.277316 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:45261
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:35873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:45261
--ranger_config_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.123.84:33061/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:05:35.632324 22056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:05:35.632679 22056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:05:35.632812 22056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:05:35.640862 22056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:05:35.641036 22056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:05:35.641144 22056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:05:35.641225 22056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:05:35.641292 22056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:05:35.651767 22056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:35873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.123.84:33061/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:45261
--ranger_config_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:45261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:05:35.653417 22056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:05:35.655443 22056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:05:35.668901 22062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:35.668968 22061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:35.668993 22056 server_base.cc:1034] running on GCE node
W20260926 15:05:35.669051 22064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:35.670437 22056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:05:35.672019 22056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:05:35.673218 22056 hybrid_clock.cc:648] HybridClock initialized: now 1790435135673171 us; error 41 us; skew 500 ppm
I20260926 15:05:35.673640 22056 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:05:35 dist-test-slave-7153 krb5kdc[21612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435135, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:35.683986 22056 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:05:35.685842 22056 webserver.cc:460] Webserver started at http://127.9.123.126:36795/ using document root <none> and password file <none>
I20260926 15:05:35.686578 22056 fs_manager.cc:359] Metadata directory not provided
I20260926 15:05:35.686697 22056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:05:35.687003 22056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 15:05:35 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435135, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for HTTP/127.9.123.84@KRBTEST.COM
I20260926 15:05:35.750232 22056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "4125662113d54580b59da35e5c60321a"
format_stamp: "Formatted at 2026-09-26 15:05:35 on dist-test-slave-7153"
server_key: "f1c0976952ed7d90613e5c58a49adc24"
server_key_iv: "a2e6974b96ba36737c01b66103c3ee45"
server_key_version: "kuduclusterkey@0"
I20260926 15:05:35.751181 22056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "4125662113d54580b59da35e5c60321a"
format_stamp: "Formatted at 2026-09-26 15:05:35 on dist-test-slave-7153"
server_key: "f1c0976952ed7d90613e5c58a49adc24"
server_key_iv: "a2e6974b96ba36737c01b66103c3ee45"
server_key_version: "kuduclusterkey@0"
I20260926 15:05:35.757357 22056 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260926 15:05:35.779469 22071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:35.781381 22056 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 15:05:35.781601 22056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "4125662113d54580b59da35e5c60321a"
format_stamp: "Formatted at 2026-09-26 15:05:35 on dist-test-slave-7153"
server_key: "f1c0976952ed7d90613e5c58a49adc24"
server_key_iv: "a2e6974b96ba36737c01b66103c3ee45"
server_key_version: "kuduclusterkey@0"
I20260926 15:05:35.781948 22056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:05:35.799037 22056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:05:35.803455 22056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:05:35.803773 22056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:05:35.822552 22056 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:45261
I20260926 15:05:35.822571 22122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:45261 every 8 connection(s)
I20260926 15:05:35.824028 22056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:05:35.825778 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 22056
I20260926 15:05:35.825990 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:05:35.826812 9709 external_mini_cluster.cc:1428] Setting key dbeabd4378c757ba4b1476728eb0f60e
I20260926 15:05:35.828954 22123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 15:05:35 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:05:35.842690 22123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a: Bootstrap starting.
I20260926 15:05:35.846341 22123 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a: Neither blocks nor log segments found. Creating new log.
I20260926 15:05:35.847689 22123 log.cc:824] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a: Log is configured to *not* fsync() on all Append() calls
I20260926 15:05:35.851100 22123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a: No bootstrap required, opened a new log
I20260926 15:05:35.851610 22126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:35.828936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45768 (local address 127.9.123.126:45261)
0926 15:05:35.829772 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:35.829785 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:35.829836 (+ 51us) server_negotiation.cc:407] Connection header received
0926 15:05:35.831555 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:35.831581 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:35.832049 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:35.832430 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:35.834166 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:35.835609 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:35.837663 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:35.838460 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:35.838621 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:35.843461 (+ 4840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:35.843471 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:35.843490 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:35.843539 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:35.846682 (+ 3143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:35.847667 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:35.847674 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:35.847684 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:35.847935 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:35.848438 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:35.848442 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:35.848445 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:35.849266 (+ 821us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:35.849508 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:35.850045 (+ 537us) server_negotiation.cc:299] Negotiation successful
0926 15:05:35.850484 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":387,"threads_started":1}
I20260926 15:05:35.859717 22123 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4125662113d54580b59da35e5c60321a" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45261 } }
I20260926 15:05:35.860148 22123 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:05:35.860244 22123 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4125662113d54580b59da35e5c60321a, State: Initialized, Role: FOLLOWER
I20260926 15:05:35.860839 22123 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4125662113d54580b59da35e5c60321a" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45261 } }
I20260926 15:05:35.861079 22123 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:05:35.861241 22123 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:05:35.861418 22123 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:05:35.863673 22123 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4125662113d54580b59da35e5c60321a" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45261 } }
I20260926 15:05:35.864233 22123 leader_election.cc:302] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4125662113d54580b59da35e5c60321a; no voters:
I20260926 15:05:35.864877 22123 leader_election.cc:288] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:05:35.865073 22128 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:05:35.866111 22128 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [term 1 LEADER]: Becoming Leader. State: Replica: 4125662113d54580b59da35e5c60321a, State: Running, Role: LEADER
I20260926 15:05:35.866720 22128 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4125662113d54580b59da35e5c60321a" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45261 } }
I20260926 15:05:35.867098 22123 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:05:35.869681 22130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4125662113d54580b59da35e5c60321a. Latest consensus state: current_term: 1 leader_uuid: "4125662113d54580b59da35e5c60321a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4125662113d54580b59da35e5c60321a" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45261 } } }
I20260926 15:05:35.869737 22129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4125662113d54580b59da35e5c60321a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4125662113d54580b59da35e5c60321a" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45261 } } }
I20260926 15:05:35.870090 22130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [sys.catalog]: This master's current role is: LEADER
I20260926 15:05:35.870090 22129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [sys.catalog]: This master's current role is: LEADER
I20260926 15:05:35.871975 22135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:05:35.877526 22135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:05:35.884500 22123 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260926 15:05:35.888254 22135 catalog_manager.cc:1347] Generated new cluster ID: ffd293a8c63547a8acb86ce1a7200823
I20260926 15:05:35.888387 22135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:05:35.907038 22135 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:05:35.909364 22135 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:05:35.930882 22135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a: Generated new TSK 0
I20260926 15:05:35.932171 22135 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.123.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
I20260926 15:05:36.637861 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.123.84:33061/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:45261
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:35873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 15:05:36 dist-test-slave-7153 krb5kdc[21612](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790435136, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260926 15:05:37.092970 22168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:05:37.093449 22168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:05:37.093670 22168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:05:37.104029 22168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:05:37.104154 22168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:05:37.104305 22168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:05:37.116022 22168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:35873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.123.84:33061/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:45261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:05:37.117682 22168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:05:37.119092 22168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:05:37.138628 22179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:37.141547 22181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:37.142336 22168 server_base.cc:1034] running on GCE node
W20260926 15:05:37.142671 22178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:37.143510 22168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:05:37.144665 22168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:05:37.145926 22168 hybrid_clock.cc:648] HybridClock initialized: now 1790435137145862 us; error 61 us; skew 500 ppm
I20260926 15:05:37.146315 22168 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:05:37 dist-test-slave-7153 krb5kdc[21612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435137, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:37.149967 22168 init.cc:377] Logged in from keytab as kudu/127.9.123.65@KRBTEST.COM (short username kudu)
I20260926 15:05:37.153432 22168 webserver.cc:460] Webserver started at http://127.9.123.65:35693/ using document root <none> and password file <none>
I20260926 15:05:37.154381 22168 fs_manager.cc:359] Metadata directory not provided
I20260926 15:05:37.154541 22168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:05:37.155093 22168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 15:05:37 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435137, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for HTTP/127.9.123.84@KRBTEST.COM
I20260926 15:05:37.174912 22168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "2addb331c3b74e0faa92ca2274455249"
format_stamp: "Formatted at 2026-09-26 15:05:37 on dist-test-slave-7153"
server_key: "e00b49f909d3ca7ffe030a584f481b9d"
server_key_iv: "f298df5ff1c264a4bbe8e2588ddd713b"
server_key_version: "kuduclusterkey@0"
I20260926 15:05:37.176251 22168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "2addb331c3b74e0faa92ca2274455249"
format_stamp: "Formatted at 2026-09-26 15:05:37 on dist-test-slave-7153"
server_key: "e00b49f909d3ca7ffe030a584f481b9d"
server_key_iv: "f298df5ff1c264a4bbe8e2588ddd713b"
server_key_version: "kuduclusterkey@0"
I20260926 15:05:37.184482 22168 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.004s
I20260926 15:05:37.230587 22188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:37.232961 22168 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.004s sys 0.008s
I20260926 15:05:37.233242 22168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "2addb331c3b74e0faa92ca2274455249"
format_stamp: "Formatted at 2026-09-26 15:05:37 on dist-test-slave-7153"
server_key: "e00b49f909d3ca7ffe030a584f481b9d"
server_key_iv: "f298df5ff1c264a4bbe8e2588ddd713b"
server_key_version: "kuduclusterkey@0"
I20260926 15:05:37.233556 22168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:05:37.269759 22168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:05:37.274689 22168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:05:37.275094 22168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:05:37.277024 22168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:05:37.278656 22168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:05:37.278743 22168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:37.278893 22168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:05:37.278947 22168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:37.336175 22168 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:46163
I20260926 15:05:37.336194 22300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:46163 every 8 connection(s)
I20260926 15:05:37.337895 22168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:05:37.341640 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 22168
I20260926 15:05:37.341857 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:05:37.342674 9709 external_mini_cluster.cc:1428] Setting key ca2163d323f9e055d4292072656231b7
Sep 26 15:05:37 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435137, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:05:37.408680 22304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:37.342402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:57545 (local address 127.9.123.126:45261)
0926 15:05:37.378009 (+ 35607us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:37.378017 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:37.380578 (+ 2561us) server_negotiation.cc:407] Connection header received
0926 15:05:37.383812 (+ 3234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:37.383820 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:37.383965 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:37.384062 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:37.387250 (+ 3188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:37.388443 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:37.391187 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:37.391876 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:37.391985 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:37.396912 (+ 4927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:37.396921 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:37.396927 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:37.396985 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:37.400565 (+ 3580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:37.405293 (+ 4728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:37.405301 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:37.405305 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:37.405547 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:37.406280 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:37.406285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:37.406289 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:37.406691 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:37.406859 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:37.407999 (+ 1140us) server_negotiation.cc:299] Negotiation successful
0926 15:05:37.408279 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35422,"thread_start_us":35621,"threads_started":1}
I20260926 15:05:37.411341 22303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:37.341691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45261 (local address 127.9.123.65:57545)
0926 15:05:37.378871 (+ 37180us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:37.378956 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:37.382168 (+ 3212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:37.385475 (+ 3307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:37.385486 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:37.386209 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:05:37.386982 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:37.387012 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:37.388716 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:37.388733 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:37.390966 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:37.390987 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:37.392374 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:37.392381 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:37.392700 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:37.393739 (+ 1039us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:05:37.393775 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:05:37.396512 (+ 2737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:05:37.400799 (+ 4287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:37.400821 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:37.400852 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:37.401573 (+ 721us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:37.405756 (+ 4183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:37.405762 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:37.405766 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:37.406118 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:37.406978 (+ 860us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:05:37.406988 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:05:37.407506 (+ 518us) client_negotiation.cc:769] Sending connection context
0926 15:05:37.409168 (+ 1662us) client_negotiation.cc:240] Negotiation successful
0926 15:05:37.409846 (+ 678us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36763,"thread_start_us":40824,"threads_started":1}
Sep 26 15:05:37 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435136, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for HTTP/127.9.123.84@KRBTEST.COM
I20260926 15:05:37.414528 22301 heartbeater.cc:347] Connected to a master server at 127.9.123.126:45261
I20260926 15:05:37.414899 22301 heartbeater.cc:464] Registering TS with master...
I20260926 15:05:37.415799 22301 heartbeater.cc:511] Master 127.9.123.126:45261 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.123.66@KRBTEST.COM; defaulting to no policy
I20260926 15:05:37.418776 22088 ts_manager.cc:194] Registered new tserver with Master: 2addb331c3b74e0faa92ca2274455249 (127.9.123.65:46163)
I20260926 15:05:37.421048 22088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.65@KRBTEST.COM'} at 127.9.123.65:57545
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
W20260926 15:05:37.470480 9709 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.9.123.66: real 0.125s user 0.006s sys 0.009s
I20260926 15:05:37.471444 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.123.84:33061/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:45261
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:35873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 15:05:37 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435136, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for HTTP/127.9.123.84@KRBTEST.COM
W20260926 15:05:38.040577 22307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:05:38.041179 22307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:05:38.041394 22307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:05:38.053413 22307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:05:38.053684 22307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:05:38.053951 22307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:05:38.071307 22307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:35873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.123.84:33061/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:45261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:05:38.073779 22307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:05:38.076274 22307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:05:38.116544 22315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:38.121636 22314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:38.125185 22318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:38.125814 22307 server_base.cc:1034] running on GCE node
I20260926 15:05:38.127108 22307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:05:38.133673 22307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:05:38.135403 22307 hybrid_clock.cc:648] HybridClock initialized: now 1790435138135272 us; error 138 us; skew 500 ppm
I20260926 15:05:38.135967 22307 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:05:38 dist-test-slave-7153 krb5kdc[21612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435138, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:38.141671 22307 init.cc:377] Logged in from keytab as kudu/127.9.123.66@KRBTEST.COM (short username kudu)
I20260926 15:05:38.163221 22307 webserver.cc:460] Webserver started at http://127.9.123.66:43739/ using document root <none> and password file <none>
I20260926 15:05:38.164443 22307 fs_manager.cc:359] Metadata directory not provided
I20260926 15:05:38.164711 22307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:05:38.165220 22307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 15:05:38 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435138, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for HTTP/127.9.123.84@KRBTEST.COM
I20260926 15:05:38.211485 22307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "503b1ffe2f3c415fbabfb9069bce5854"
format_stamp: "Formatted at 2026-09-26 15:05:38 on dist-test-slave-7153"
server_key: "eee74a0a14b5a22591464777e6c8a206"
server_key_iv: "77adc4fe55f0f7a8754d80e273119843"
server_key_version: "kuduclusterkey@0"
I20260926 15:05:38.220752 22307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "503b1ffe2f3c415fbabfb9069bce5854"
format_stamp: "Formatted at 2026-09-26 15:05:38 on dist-test-slave-7153"
server_key: "eee74a0a14b5a22591464777e6c8a206"
server_key_iv: "77adc4fe55f0f7a8754d80e273119843"
server_key_version: "kuduclusterkey@0"
I20260926 15:05:38.236097 22307 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.006s sys 0.000s
I20260926 15:05:38.265859 22331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:38.271602 22307 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.002s sys 0.001s
I20260926 15:05:38.271874 22307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "503b1ffe2f3c415fbabfb9069bce5854"
format_stamp: "Formatted at 2026-09-26 15:05:38 on dist-test-slave-7153"
server_key: "eee74a0a14b5a22591464777e6c8a206"
server_key_iv: "77adc4fe55f0f7a8754d80e273119843"
server_key_version: "kuduclusterkey@0"
I20260926 15:05:38.272166 22307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:05:38.338181 22307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:05:38.343323 22307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:05:38.343626 22307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:05:38.344657 22307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:05:38.346158 22307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:05:38.346238 22307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:38.346356 22307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:05:38.346423 22307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:38.386071 22307 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:34289
I20260926 15:05:38.386142 22443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:34289 every 8 connection(s)
I20260926 15:05:38.387506 22307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:05:38.388329 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 22307
I20260926 15:05:38.388499 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:05:38.389007 9709 external_mini_cluster.cc:1428] Setting key c4cd60203e9f880fbb6c6d5dcce2882c
Sep 26 15:05:38 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435138, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.123.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.67@KRBTEST.COM" created.
I20260926 15:05:38.420724 22446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:38.391090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:40789 (local address 127.9.123.126:45261)
0926 15:05:38.403884 (+ 12794us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:38.403891 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:38.403917 (+ 26us) server_negotiation.cc:407] Connection header received
0926 15:05:38.405798 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:38.405805 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:38.405909 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:38.406006 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:38.407690 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:38.408497 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:38.410208 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:38.410633 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:38.410754 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:38.414223 (+ 3469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:38.414228 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:38.414232 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:38.414275 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:38.416999 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:38.417900 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:38.417905 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:38.417907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:38.418035 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:38.418604 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:38.418610 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:38.418614 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:38.419007 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:38.419157 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:38.420083 (+ 926us) server_negotiation.cc:299] Negotiation successful
0926 15:05:38.420335 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12646,"thread_start_us":12831,"threads_started":1}
I20260926 15:05:38.421984 22447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:38.391780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45261 (local address 127.9.123.66:40789)
0926 15:05:38.402967 (+ 11187us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:38.403037 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:38.405464 (+ 2427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:38.406316 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:38.406325 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:38.406840 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:05:38.407501 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:38.407521 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:38.408666 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:38.408670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:38.410055 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:38.410069 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:38.410873 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:38.410881 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:38.411184 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:38.411857 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:05:38.411881 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:05:38.413986 (+ 2105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:05:38.417207 (+ 3221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:38.417218 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:38.417237 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:38.417729 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:38.418144 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:38.418149 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:38.418152 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:38.418455 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:38.419152 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:05:38.419160 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:05:38.419624 (+ 464us) client_negotiation.cc:769] Sending connection context
0926 15:05:38.419939 (+ 315us) client_negotiation.cc:240] Negotiation successful
0926 15:05:38.420522 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10745,"thread_start_us":12080,"threads_started":1}
I20260926 15:05:38.423751 22444 heartbeater.cc:347] Connected to a master server at 127.9.123.126:45261
I20260926 15:05:38.424151 22444 heartbeater.cc:464] Registering TS with master...
I20260926 15:05:38.425037 22444 heartbeater.cc:511] Master 127.9.123.126:45261 requested a full tablet report, sending...
I20260926 15:05:38.425717 22301 heartbeater.cc:503] Master 127.9.123.126:45261 was elected leader, sending a full tablet report...
I20260926 15:05:38.426940 22087 ts_manager.cc:194] Registered new tserver with Master: 503b1ffe2f3c415fbabfb9069bce5854 (127.9.123.66:34289)
I20260926 15:05:38.428001 22087 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.66@KRBTEST.COM'} at 127.9.123.66:40789
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
I20260926 15:05:38.453130 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.123.84:33061/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:45261
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:35873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:05:38.800734 22450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:05:38.801048 22450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:05:38.801183 22450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:05:38.808547 22450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:05:38.808689 22450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:05:38.808835 22450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:05:38.819123 22450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:35873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.123.84:33061/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:45261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:05:38.820498 22450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:05:38.821973 22450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:05:38.837517 22455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:38.837637 22458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:38.837671 22456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:38.838274 22450 server_base.cc:1034] running on GCE node
I20260926 15:05:38.838912 22450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:05:38.840013 22450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:05:38.841221 22450 hybrid_clock.cc:648] HybridClock initialized: now 1790435138841143 us; error 67 us; skew 500 ppm
I20260926 15:05:38.841696 22450 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:05:38 dist-test-slave-7153 krb5kdc[21612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435138, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:38.845177 22450 init.cc:377] Logged in from keytab as kudu/127.9.123.67@KRBTEST.COM (short username kudu)
I20260926 15:05:38.846738 22450 webserver.cc:460] Webserver started at http://127.9.123.67:35551/ using document root <none> and password file <none>
I20260926 15:05:38.847462 22450 fs_manager.cc:359] Metadata directory not provided
I20260926 15:05:38.847589 22450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:05:38.847904 22450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 26 15:05:38 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435138, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for HTTP/127.9.123.84@KRBTEST.COM
I20260926 15:05:38.864537 22450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "f400a972edb94463a2b6ec56af27d06d"
format_stamp: "Formatted at 2026-09-26 15:05:38 on dist-test-slave-7153"
server_key: "f76cf6eba67ba39cd083c24c65b6713e"
server_key_iv: "acaefaef1bc4245953c68b9c82a7eec5"
server_key_version: "kuduclusterkey@0"
I20260926 15:05:38.865466 22450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "f400a972edb94463a2b6ec56af27d06d"
format_stamp: "Formatted at 2026-09-26 15:05:38 on dist-test-slave-7153"
server_key: "f76cf6eba67ba39cd083c24c65b6713e"
server_key_iv: "acaefaef1bc4245953c68b9c82a7eec5"
server_key_version: "kuduclusterkey@0"
I20260926 15:05:38.871686 22450 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 15:05:38.887058 22465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:38.888675 22450 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 15:05:38.888849 22450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "f400a972edb94463a2b6ec56af27d06d"
format_stamp: "Formatted at 2026-09-26 15:05:38 on dist-test-slave-7153"
server_key: "f76cf6eba67ba39cd083c24c65b6713e"
server_key_iv: "acaefaef1bc4245953c68b9c82a7eec5"
server_key_version: "kuduclusterkey@0"
I20260926 15:05:38.889065 22450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:05:38.910996 22450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:05:38.915549 22450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:05:38.915856 22450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:05:38.916893 22450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:05:38.918406 22450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:05:38.918478 22450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:38.918599 22450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:05:38.918655 22450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:38.950114 22450 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:40743
I20260926 15:05:38.950131 22577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:40743 every 8 connection(s)
I20260926 15:05:38.951428 22450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:05:38.958660 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 22450
I20260926 15:05:38.958827 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:05:38.959331 9709 external_mini_cluster.cc:1428] Setting key dd46dcc18c5189b6faa9e8664f9c5b14
Sep 26 15:05:38 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435138, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:05:38.973080 22580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:38.953984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:49653 (local address 127.9.123.126:45261)
0926 15:05:38.954450 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:38.954456 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:38.955508 (+ 1052us) server_negotiation.cc:407] Connection header received
0926 15:05:38.957299 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:38.957304 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:38.957415 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:38.957515 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:38.959179 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:38.960150 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:38.962106 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:38.962705 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:38.962802 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:38.966396 (+ 3594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:38.966402 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:38.966405 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:38.966519 (+ 114us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:38.969491 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:38.970371 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:38.970375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:38.970377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:38.970511 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:38.971091 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:38.971095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:38.971098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:38.971401 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:38.971538 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:38.972343 (+ 805us) server_negotiation.cc:299] Negotiation successful
0926 15:05:38.972596 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":208,"threads_started":1}
I20260926 15:05:38.974270 22581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:38.954638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45261 (local address 127.9.123.67:49653)
0926 15:05:38.955341 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:38.955386 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:38.956995 (+ 1609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:38.957804 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:38.957814 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:38.958279 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:05:38.958977 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:38.958999 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:38.960338 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:38.960344 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:38.961962 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:38.961978 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:38.962937 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:38.962942 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:38.963157 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:38.963830 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:05:38.963854 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:05:38.965975 (+ 2121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:05:38.969688 (+ 3713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:38.969729 (+ 41us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:38.969745 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:38.970249 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:38.970655 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:38.970661 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:38.970664 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:38.970933 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:38.971549 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:05:38.971556 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:05:38.971978 (+ 422us) client_negotiation.cc:769] Sending connection context
0926 15:05:38.972221 (+ 243us) client_negotiation.cc:240] Negotiation successful
0926 15:05:38.972749 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":271,"threads_started":1}
I20260926 15:05:38.976482 22578 heartbeater.cc:347] Connected to a master server at 127.9.123.126:45261
I20260926 15:05:38.977078 22578 heartbeater.cc:464] Registering TS with master...
I20260926 15:05:38.978312 22578 heartbeater.cc:511] Master 127.9.123.126:45261 requested a full tablet report, sending...
I20260926 15:05:38.980314 22087 ts_manager.cc:194] Registered new tserver with Master: f400a972edb94463a2b6ec56af27d06d (127.9.123.67:40743)
I20260926 15:05:38.981262 22087 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.67@KRBTEST.COM'} at 127.9.123.67:49653
I20260926 15:05:38.989315 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 15:05:38.991145 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 22168
Sep 26 15:05:38 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.65@KRBTEST.COM
I20260926 15:05:39.012310 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:38.992268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56368 (local address 127.9.123.65:46163)
0926 15:05:38.992851 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:38.992858 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:38.992886 (+ 28us) server_negotiation.cc:407] Connection header received
0926 15:05:38.992960 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:38.992969 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:38.993273 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:38.993444 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:38.995429 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:38.996939 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:38.998961 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:38.999737 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:38.999871 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:39.005304 (+ 5433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:39.005327 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:39.005348 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:39.005410 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:39.009183 (+ 3773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:39.009965 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:39.009975 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:39.009979 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:39.010180 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:39.010624 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:39.010630 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:39.010634 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:39.011035 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:39.011336 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:39.011687 (+ 351us) server_negotiation.cc:299] Negotiation successful
0926 15:05:39.011946 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":273,"threads_started":1}
I20260926 15:05:39.013827 22296 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:05:39.098500 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 22168
I20260926 15:05:39.111083 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 22307
Sep 26 15:05:39 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.66@KRBTEST.COM
I20260926 15:05:39.128271 22585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:39.112104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52886 (local address 127.9.123.66:34289)
0926 15:05:39.112627 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:39.112634 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:39.112660 (+ 26us) server_negotiation.cc:407] Connection header received
0926 15:05:39.112722 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:39.112730 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:39.112959 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:39.113134 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:39.114378 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:39.115381 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:39.116738 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:39.117403 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:39.117514 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:39.121810 (+ 4296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:39.121823 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:39.121839 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:39.121879 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:39.124858 (+ 2979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:39.126002 (+ 1144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:39.126011 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:39.126013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:39.126177 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:39.126666 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:39.126671 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:39.126673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:39.127113 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:39.127364 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:39.127708 (+ 344us) server_negotiation.cc:299] Negotiation successful
0926 15:05:39.127974 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":284,"threads_started":1}
I20260926 15:05:39.129647 22439 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:05:39.211746 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 22307
I20260926 15:05:39.224601 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 22450
Sep 26 15:05:39 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.67@KRBTEST.COM
I20260926 15:05:39.245986 22587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:39.225624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34810 (local address 127.9.123.67:40743)
0926 15:05:39.226204 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:39.226213 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:39.226241 (+ 28us) server_negotiation.cc:407] Connection header received
0926 15:05:39.226302 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:39.226309 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:39.226554 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:39.226701 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:39.228113 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:39.229275 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:39.230769 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:39.231396 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:39.231529 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:39.238554 (+ 7025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:39.238579 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:39.238601 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:39.238753 (+ 152us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:39.242402 (+ 3649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:39.243360 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:39.243369 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:39.243372 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:39.243539 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:39.244141 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:39.244146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:39.244148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:39.244559 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:39.244817 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:39.245319 (+ 502us) server_negotiation.cc:299] Negotiation successful
0926 15:05:39.245576 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":286,"threads_started":1}
I20260926 15:05:39.247370 22573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:05:39.331312 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 22450
I20260926 15:05:39.343976 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 22056
I20260926 15:05:39.345428 22118 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:05:39.454066 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 22056
I20260926 15:05:39.471644 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:45261
--webserver_interface=127.9.123.126
--webserver_port=36795
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:35873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:45261
--ranger_config_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.123.84:33061/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.123.84:33061/kms with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:05:39.808183 22590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:05:39.808503 22590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:05:39.808611 22590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:05:39.816461 22590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:05:39.816613 22590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:05:39.816686 22590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:05:39.816769 22590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:05:39.816835 22590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:05:39.827958 22590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:35873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.123.84:33061/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:45261
--ranger_config_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:45261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=36795
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:05:39.829453 22590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:05:39.830905 22590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:05:39.843369 22595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:39.843403 22598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:39.843429 22596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:39.844043 22590 server_base.cc:1034] running on GCE node
I20260926 15:05:39.844918 22590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:05:39.846789 22590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:05:39.848137 22590 hybrid_clock.cc:648] HybridClock initialized: now 1790435139848067 us; error 68 us; skew 500 ppm
I20260926 15:05:39.848547 22590 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:05:39 dist-test-slave-7153 krb5kdc[21612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435139, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:39.851889 22590 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:05:39.853582 22590 webserver.cc:460] Webserver started at http://127.9.123.126:36795/ using document root <none> and password file <none>
I20260926 15:05:39.854285 22590 fs_manager.cc:359] Metadata directory not provided
I20260926 15:05:39.854414 22590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 15:05:39 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435139, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for HTTP/127.9.123.84@KRBTEST.COM
I20260926 15:05:39.922923 22590 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 15:05:39.927452 22606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:39.929646 22590 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260926 15:05:39.929862 22590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "4125662113d54580b59da35e5c60321a"
format_stamp: "Formatted at 2026-09-26 15:05:35 on dist-test-slave-7153"
server_key: "f1c0976952ed7d90613e5c58a49adc24"
server_key_iv: "a2e6974b96ba36737c01b66103c3ee45"
server_key_version: "kuduclusterkey@0"
I20260926 15:05:39.931015 22590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:05:39.961189 22590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:05:39.965906 22590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:05:39.966219 22590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:05:39.987252 22590 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:45261
I20260926 15:05:39.987262 22657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:45261 every 8 connection(s)
I20260926 15:05:39.988782 22590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:05:39.996002 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 22590
I20260926 15:05:39.996131 22658 sys_catalog.cc:263] Verifying existing consensus state
I20260926 15:05:39.996850 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.123.84:33061/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:46163
--local_ip_for_outbound_sockets=127.9.123.65
--tserver_master_addrs=127.9.123.126:45261
--webserver_port=35693
--webserver_interface=127.9.123.65
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:35873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.123.84:33061/kms with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:05:39.999744 22658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a: Bootstrap starting.
I20260926 15:05:40.024535 22658 log.cc:824] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a: Log is configured to *not* fsync() on all Append() calls
I20260926 15:05:40.038892 22658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 15:05:40.039593 22658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a: Bootstrap complete.
I20260926 15:05:40.052949 22658 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4125662113d54580b59da35e5c60321a" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45261 } }
I20260926 15:05:40.054317 22658 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4125662113d54580b59da35e5c60321a, State: Initialized, Role: FOLLOWER
I20260926 15:05:40.055177 22658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4125662113d54580b59da35e5c60321a" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45261 } }
I20260926 15:05:40.055477 22658 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:05:40.055608 22658 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:05:40.055800 22658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [term 1 FOLLOWER]: Advancing to term 2
I20260926 15:05:40.061604 22658 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4125662113d54580b59da35e5c60321a" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45261 } }
I20260926 15:05:40.062276 22658 leader_election.cc:302] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4125662113d54580b59da35e5c60321a; no voters:
I20260926 15:05:40.062999 22658 leader_election.cc:288] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 15:05:40.063159 22662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [term 2 FOLLOWER]: Leader election won for term 2
I20260926 15:05:40.064414 22662 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [term 2 LEADER]: Becoming Leader. State: Replica: 4125662113d54580b59da35e5c60321a, State: Running, Role: LEADER
I20260926 15:05:40.065007 22662 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4125662113d54580b59da35e5c60321a" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45261 } }
I20260926 15:05:40.065548 22658 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:05:40.067467 22663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4125662113d54580b59da35e5c60321a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4125662113d54580b59da35e5c60321a" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45261 } } }
I20260926 15:05:40.067863 22663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [sys.catalog]: This master's current role is: LEADER
I20260926 15:05:40.069510 22667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:05:40.069284 22664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4125662113d54580b59da35e5c60321a. Latest consensus state: current_term: 2 leader_uuid: "4125662113d54580b59da35e5c60321a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4125662113d54580b59da35e5c60321a" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45261 } } }
I20260926 15:05:40.069674 22664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a [sys.catalog]: This master's current role is: LEADER
I20260926 15:05:40.075707 22667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:05:40.077502 22667 catalog_manager.cc:1259] Loaded cluster ID: ffd293a8c63547a8acb86ce1a7200823
I20260926 15:05:40.077725 22667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:05:40.082574 22667 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:05:40.084512 22667 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a: Loaded TSK: 0
I20260926 15:05:40.086455 22667 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:05:40.098138 22658 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260926 15:05:40.487578 22660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:05:40.488163 22660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:05:40.488406 22660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:05:40.500727 22660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:05:40.500965 22660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:05:40.501194 22660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:05:40.517195 22660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:35873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.123.84:33061/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:46163
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=35693
--tserver_master_addrs=127.9.123.126:45261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:05:40.518934 22660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:05:40.520783 22660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:05:40.567739 22696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:40.572986 22695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:40.573907 22660 server_base.cc:1034] running on GCE node
W20260926 15:05:40.574765 22698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:40.575609 22660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:05:40.585501 22660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:05:40.588999 22660 hybrid_clock.cc:648] HybridClock initialized: now 1790435140588905 us; error 96 us; skew 500 ppm
I20260926 15:05:40.589514 22660 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:05:40 dist-test-slave-7153 krb5kdc[21612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435140, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:40.593936 22660 init.cc:377] Logged in from keytab as kudu/127.9.123.65@KRBTEST.COM (short username kudu)
I20260926 15:05:40.596335 22660 webserver.cc:460] Webserver started at http://127.9.123.65:35693/ using document root <none> and password file <none>
I20260926 15:05:40.597415 22660 fs_manager.cc:359] Metadata directory not provided
I20260926 15:05:40.597586 22660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 15:05:40 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435140, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for HTTP/127.9.123.84@KRBTEST.COM
I20260926 15:05:40.620718 22660 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 15:05:40.632057 22707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:40.633965 22660 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.003s sys 0.008s
I20260926 15:05:40.634173 22660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "2addb331c3b74e0faa92ca2274455249"
format_stamp: "Formatted at 2026-09-26 15:05:37 on dist-test-slave-7153"
server_key: "e00b49f909d3ca7ffe030a584f481b9d"
server_key_iv: "f298df5ff1c264a4bbe8e2588ddd713b"
server_key_version: "kuduclusterkey@0"
I20260926 15:05:40.635280 22660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:05:40.660224 22660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:05:40.665730 22660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:05:40.666217 22660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:05:40.668747 22660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:05:40.670485 22660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:05:40.670657 22660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260926 15:05:40.670791 22660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:05:40.670866 22660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:40.736181 22660 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:46163
I20260926 15:05:40.736303 22819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:46163 every 8 connection(s)
I20260926 15:05:40.738122 22660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:05:40.739281 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 22660
I20260926 15:05:40.740190 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.123.84:33061/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:34289
--local_ip_for_outbound_sockets=127.9.123.66
--tserver_master_addrs=127.9.123.126:45261
--webserver_port=43739
--webserver_interface=127.9.123.66
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:35873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.123.84:33061/kms with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 15:05:40 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435140, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:05:40.808099 22823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:40.754316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:44139 (local address 127.9.123.126:45261)
0926 15:05:40.769317 (+ 15001us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:40.769345 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:40.774135 (+ 4790us) server_negotiation.cc:407] Connection header received
0926 15:05:40.777603 (+ 3468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:40.777627 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:40.778277 (+ 650us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:40.778813 (+ 536us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:40.781752 (+ 2939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:40.783878 (+ 2126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:40.786645 (+ 2767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:40.787729 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:40.787913 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:40.798415 (+ 10502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:40.798426 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:40.798446 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:40.798509 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:40.802275 (+ 3766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:40.803399 (+ 1124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:40.803409 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:40.803417 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:40.803638 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:40.804214 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:40.804220 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:40.804223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:40.804883 (+ 660us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:40.805070 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:40.806388 (+ 1318us) server_negotiation.cc:299] Negotiation successful
0926 15:05:40.806847 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14573,"thread_start_us":14910,"threads_started":1}
I20260926 15:05:40.810252 22824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:40.754513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45261 (local address 127.9.123.65:44139)
0926 15:05:40.773873 (+ 19360us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:40.773945 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:40.776616 (+ 2671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:40.779721 (+ 3105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:40.779736 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:40.780437 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:05:40.781409 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:40.781445 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:40.784160 (+ 2715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:40.784167 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:40.786443 (+ 2276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:40.786462 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:40.788369 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:40.788377 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:40.788715 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:40.795056 (+ 6341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:05:40.795125 (+ 69us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:05:40.798032 (+ 2907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:05:40.802529 (+ 4497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:40.802538 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:40.802554 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:40.803200 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:40.803794 (+ 594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:40.803800 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:40.803803 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:40.804068 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:40.805187 (+ 1119us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:05:40.805195 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:05:40.805586 (+ 391us) client_negotiation.cc:769] Sending connection context
0926 15:05:40.805910 (+ 324us) client_negotiation.cc:240] Negotiation successful
0926 15:05:40.809019 (+ 3109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18979,"thread_start_us":18774,"threads_started":1}
I20260926 15:05:40.813903 22820 heartbeater.cc:347] Connected to a master server at 127.9.123.126:45261
I20260926 15:05:40.814242 22820 heartbeater.cc:464] Registering TS with master...
I20260926 15:05:40.815112 22820 heartbeater.cc:511] Master 127.9.123.126:45261 requested a full tablet report, sending...
W20260926 15:05:40.818665 22820 heartbeater.cc:644] Failed to heartbeat to 127.9.123.126:45261 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 15:05:40.818907 22820 heartbeater.cc:464] Registering TS with master...
I20260926 15:05:40.819204 22820 heartbeater.cc:511] Master 127.9.123.126:45261 requested a full tablet report, sending...
I20260926 15:05:40.820746 22820 heartbeater.cc:464] Registering TS with master...
I20260926 15:05:40.821199 22820 heartbeater.cc:511] Master 127.9.123.126:45261 requested a full tablet report, sending...
W20260926 15:05:40.822588 22820 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 15:05:41.227702 22821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:05:41.228060 22821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:05:41.228217 22821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:05:41.236919 22821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:05:41.237067 22821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:05:41.237265 22821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:05:41.249051 22821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:35873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.123.84:33061/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:34289
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=43739
--tserver_master_addrs=127.9.123.126:45261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:05:41.250902 22821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:05:41.252898 22821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:05:41.293949 22839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:41.293927 22841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:41.294605 22838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:41.296710 22821 server_base.cc:1034] running on GCE node
I20260926 15:05:41.297580 22821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:05:41.306995 22821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:05:41.308343 22821 hybrid_clock.cc:648] HybridClock initialized: now 1790435141308267 us; error 74 us; skew 500 ppm
I20260926 15:05:41.308780 22821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:05:41 dist-test-slave-7153 krb5kdc[21612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435141, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:41.313009 22821 init.cc:377] Logged in from keytab as kudu/127.9.123.66@KRBTEST.COM (short username kudu)
I20260926 15:05:41.324875 22821 webserver.cc:460] Webserver started at http://127.9.123.66:43739/ using document root <none> and password file <none>
I20260926 15:05:41.325977 22821 fs_manager.cc:359] Metadata directory not provided
I20260926 15:05:41.326169 22821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 15:05:41 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435141, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for HTTP/127.9.123.84@KRBTEST.COM
I20260926 15:05:41.358248 22821 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.004s sys 0.000s
I20260926 15:05:41.367018 22851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:41.369272 22821 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.004s sys 0.005s
I20260926 15:05:41.369493 22821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "503b1ffe2f3c415fbabfb9069bce5854"
format_stamp: "Formatted at 2026-09-26 15:05:38 on dist-test-slave-7153"
server_key: "eee74a0a14b5a22591464777e6c8a206"
server_key_iv: "77adc4fe55f0f7a8754d80e273119843"
server_key_version: "kuduclusterkey@0"
I20260926 15:05:41.371091 22821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Sep 26 15:05:41 dist-test-slave-7153 krb5kdc[21612](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790435141, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:41.392630 22821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:05:41.398885 22821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:05:41.399224 22821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:05:41.403524 22821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:05:41.406618 22821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:05:41.406754 22821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:41.406916 22821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:05:41.406996 22821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:41.468780 22821 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:34289
I20260926 15:05:41.468830 22964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:34289 every 8 connection(s)
I20260926 15:05:41.470819 22821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:05:41.479382 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 22821
I20260926 15:05:41.480252 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.123.84:33061/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:40743
--local_ip_for_outbound_sockets=127.9.123.67
--tserver_master_addrs=127.9.123.126:45261
--webserver_port=35551
--webserver_interface=127.9.123.67
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:35873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.123.84:33061/kms with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 15:05:41 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435141, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:05:41.497650 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:41.475091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:34135 (local address 127.9.123.126:45261)
0926 15:05:41.477995 (+ 2904us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:41.478005 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:41.481648 (+ 3643us) server_negotiation.cc:407] Connection header received
0926 15:05:41.481734 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:41.481739 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:41.481846 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:41.481950 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:41.483762 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:41.484695 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:41.486514 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:41.486926 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:41.487041 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:41.491065 (+ 4024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:41.491071 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:41.491074 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:41.491114 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:41.494179 (+ 3065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:41.495012 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:41.495018 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:41.495022 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:41.495154 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:41.495621 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:41.495626 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:41.495629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:41.495919 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:41.496059 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:41.496985 (+ 926us) server_negotiation.cc:299] Negotiation successful
0926 15:05:41.497259 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2684,"thread_start_us":2583,"threads_started":1}
I20260926 15:05:41.498718 22967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:41.474544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45261 (local address 127.9.123.66:34135)
0926 15:05:41.478576 (+ 4032us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:41.478627 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:41.481211 (+ 2584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:41.482373 (+ 1162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:41.482383 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:41.482877 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:05:41.483570 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:41.483591 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:41.484851 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:41.484855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:41.486358 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:41.486372 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:41.487174 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:41.487182 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:41.487411 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:41.488060 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:05:41.488088 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:05:41.490771 (+ 2683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:05:41.494355 (+ 3584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:41.494365 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:41.494387 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:41.494865 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:41.495265 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:41.495270 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:41.495274 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:41.495495 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:41.496046 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:05:41.496056 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:05:41.496579 (+ 523us) client_negotiation.cc:769] Sending connection context
0926 15:05:41.496853 (+ 274us) client_negotiation.cc:240] Negotiation successful
0926 15:05:41.497465 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3735,"thread_start_us":3508,"threads_started":1}
I20260926 15:05:41.500337 22965 heartbeater.cc:347] Connected to a master server at 127.9.123.126:45261
I20260926 15:05:41.500631 22965 heartbeater.cc:464] Registering TS with master...
I20260926 15:05:41.501398 22965 heartbeater.cc:511] Master 127.9.123.126:45261 requested a full tablet report, sending...
I20260926 15:05:41.503127 22623 ts_manager.cc:194] Registered new tserver with Master: 503b1ffe2f3c415fbabfb9069bce5854 (127.9.123.66:34289)
I20260926 15:05:41.504735 22623 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.66@KRBTEST.COM'} at 127.9.123.66:34135
I20260926 15:05:41.822721 22820 heartbeater.cc:464] Registering TS with master...
I20260926 15:05:41.823165 22820 heartbeater.cc:511] Master 127.9.123.126:45261 requested a full tablet report, sending...
I20260926 15:05:41.824429 22623 ts_manager.cc:194] Registered new tserver with Master: 2addb331c3b74e0faa92ca2274455249 (127.9.123.65:46163)
I20260926 15:05:41.825664 22623 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.65@KRBTEST.COM'} at 127.9.123.65:44139
Sep 26 15:05:41 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435141, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for HTTP/127.9.123.84@KRBTEST.COM
W20260926 15:05:41.932714 22969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:05:41.933027 22969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:05:41.933213 22969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:05:41.941017 22969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:05:41.941177 22969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:05:41.941340 22969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:05:41.955068 22969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:35873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.123.84:33061/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:40743
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=35551
--tserver_master_addrs=127.9.123.126:45261
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:05:41.956930 22969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:05:41.958907 22969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:05:41.975045 22977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:41.975521 22974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:41.975615 22975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:41.976213 22969 server_base.cc:1034] running on GCE node
I20260926 15:05:41.977118 22969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:05:41.978473 22969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:05:41.979745 22969 hybrid_clock.cc:648] HybridClock initialized: now 1790435141979641 us; error 85 us; skew 500 ppm
I20260926 15:05:41.980135 22969 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:05:41 dist-test-slave-7153 krb5kdc[21612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435141, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:41.983503 22969 init.cc:377] Logged in from keytab as kudu/127.9.123.67@KRBTEST.COM (short username kudu)
I20260926 15:05:41.985321 22969 webserver.cc:460] Webserver started at http://127.9.123.67:35551/ using document root <none> and password file <none>
I20260926 15:05:41.986080 22969 fs_manager.cc:359] Metadata directory not provided
I20260926 15:05:41.986225 22969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 26 15:05:41 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435141, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for HTTP/127.9.123.84@KRBTEST.COM
I20260926 15:05:42.015022 22969 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.003s
I20260926 15:05:42.018069 22985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:42.019681 22969 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:05:42.019876 22969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "f400a972edb94463a2b6ec56af27d06d"
format_stamp: "Formatted at 2026-09-26 15:05:38 on dist-test-slave-7153"
server_key: "f76cf6eba67ba39cd083c24c65b6713e"
server_key_iv: "acaefaef1bc4245953c68b9c82a7eec5"
server_key_version: "kuduclusterkey@0"
I20260926 15:05:42.021051 22969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:05:42.050529 22969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:05:42.054843 22969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:05:42.055146 22969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:05:42.056191 22969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:05:42.057716 22969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:05:42.057813 22969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:42.057928 22969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:05:42.058007 22969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:42.110918 22969 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:40743
I20260926 15:05:42.110972 23097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:40743 every 8 connection(s)
I20260926 15:05:42.112838 22969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:05:42.114138 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 22969
Sep 26 15:05:42 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435141, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:05:42.160534 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:42.134320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:60683 (local address 127.9.123.126:45261)
0926 15:05:42.134912 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:42.134918 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:42.136774 (+ 1856us) server_negotiation.cc:407] Connection header received
0926 15:05:42.140906 (+ 4132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:42.140913 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:42.141020 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:42.141157 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:42.143510 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.144732 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.147197 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.147830 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.147939 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:42.152280 (+ 4341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:42.152287 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:42.152292 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:42.152357 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:42.156613 (+ 4256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:42.157668 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:42.157673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:42.157677 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:42.157826 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:42.158363 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:42.158368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:42.158371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:42.158725 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:42.158868 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:42.159892 (+ 1024us) server_negotiation.cc:299] Negotiation successful
0926 15:05:42.160157 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":291,"threads_started":1}
I20260926 15:05:42.162855 23100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:42.115202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51790 (local address 127.9.123.126:45261)
0926 15:05:42.134100 (+ 18898us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:42.134108 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:42.135521 (+ 1413us) server_negotiation.cc:407] Connection header received
0926 15:05:42.135614 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:42.135620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:42.135781 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:42.135898 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:42.139125 (+ 3227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.140278 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.146306 (+ 6028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.146731 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.146852 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:42.150644 (+ 3792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:42.150651 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:42.150656 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:42.150703 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:42.155103 (+ 4400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:42.156611 (+ 1508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:42.156617 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:42.156620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:42.156778 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:42.158704 (+ 1926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:42.158710 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:42.158713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:42.159051 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:42.159291 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:42.162259 (+ 2968us) server_negotiation.cc:299] Negotiation successful
0926 15:05:42.162483 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18723,"thread_start_us":18917,"threads_started":1}
I20260926 15:05:42.163043 23102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:42.117720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45261 (local address 127.9.123.67:60683)
0926 15:05:42.136622 (+ 18902us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:42.136675 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:42.140576 (+ 3901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:42.141678 (+ 1102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:42.141692 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:42.142350 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:05:42.143264 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:42.143293 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.144944 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.144950 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:42.147029 (+ 2079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:42.147046 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.148303 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.148311 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:42.148624 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:42.149395 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:05:42.149427 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:05:42.151960 (+ 2533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:05:42.156797 (+ 4837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:42.156807 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:42.156828 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:42.157527 (+ 699us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:42.157983 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:42.157988 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:42.157992 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:42.158229 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:42.158975 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:05:42.158984 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:05:42.159452 (+ 468us) client_negotiation.cc:769] Sending connection context
0926 15:05:42.161039 (+ 1587us) client_negotiation.cc:240] Negotiation successful
0926 15:05:42.161691 (+ 652us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18585,"thread_start_us":18348,"threads_started":1}
I20260926 15:05:42.164636 23098 heartbeater.cc:347] Connected to a master server at 127.9.123.126:45261
I20260926 15:05:42.164980 23098 heartbeater.cc:464] Registering TS with master...
I20260926 15:05:42.166054 23098 heartbeater.cc:511] Master 127.9.123.126:45261 requested a full tablet report, sending...
I20260926 15:05:42.167940 22623 ts_manager.cc:194] Registered new tserver with Master: f400a972edb94463a2b6ec56af27d06d (127.9.123.67:40743)
I20260926 15:05:42.169183 22623 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.67@KRBTEST.COM'} at 127.9.123.67:60683
I20260926 15:05:42.180646 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 26 15:05:42 dist-test-slave-7153 krb5kdc[21612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435141, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for HTTP/127.9.123.84@KRBTEST.COM
I20260926 15:05:42.203855 23100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:42.183945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51804 (local address 127.9.123.126:45261)
0926 15:05:42.184137 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:42.184145 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:42.184731 (+ 586us) server_negotiation.cc:407] Connection header received
0926 15:05:42.184932 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:42.184938 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:42.185012 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:42.185126 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:42.187481 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.188194 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.189863 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.190565 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.190682 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:42.198060 (+ 7378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:42.198066 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:42.198070 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:42.198114 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:42.200762 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:42.201618 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:42.201623 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:42.201626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:42.201767 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:42.202167 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:42.202172 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:42.202174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:42.202479 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:42.202702 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:42.203093 (+ 391us) server_negotiation.cc:299] Negotiation successful
0926 15:05:42.203498 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 15:05:42.211699 22623 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51804:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:05:42.223107 22623 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:05:42.286713 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:42.251109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56324 (local address 127.9.123.65:46163)
0926 15:05:42.262145 (+ 11036us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:42.262155 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:42.264576 (+ 2421us) server_negotiation.cc:407] Connection header received
0926 15:05:42.264690 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:42.264698 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:42.264966 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:42.265208 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:05:42.278066 (+ 12858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.279799 (+ 1733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.283078 (+ 3279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.284668 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.284818 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:42.285085 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:42.286122 (+ 1037us) server_negotiation.cc:299] Negotiation successful
0926 15:05:42.286327 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10843,"thread_start_us":10656,"threads_started":1}
I20260926 15:05:42.287943 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:42.250393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:46163 (local address 127.0.0.1:56324)
0926 15:05:42.264225 (+ 13832us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:42.264252 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:42.264432 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:42.265537 (+ 1105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:42.265545 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:42.265579 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:05:42.277647 (+ 12068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:42.277671 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.280017 (+ 2346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.280029 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:42.282850 (+ 2821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:42.282868 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.285339 (+ 2471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.285347 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:42.285590 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:42.285657 (+ 67us) client_negotiation.cc:769] Sending connection context
0926 15:05:42.287242 (+ 1585us) client_negotiation.cc:240] Negotiation successful
0926 15:05:42.287495 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13689,"mutex_wait_us":4,"spinlock_wait_cycles":896,"thread_start_us":12403,"threads_started":1}
I20260926 15:05:42.293051 22755 tablet_service.cc:1467] Processing CreateTablet for tablet 7d386b307bba4fb9bbe4153fdc521575 (DEFAULT_TABLE table=test-table [id=e74177d8cd114731966c00d7cc7c1871]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:05:42.295895 22755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d386b307bba4fb9bbe4153fdc521575. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:05:42.298558 23115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:42.244512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43902 (local address 127.9.123.66:34289)
0926 15:05:42.261589 (+ 17077us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:42.261601 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:42.263408 (+ 1807us) server_negotiation.cc:407] Connection header received
0926 15:05:42.263536 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:42.263546 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:42.263860 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:42.264073 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:05:42.278529 (+ 14456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.280111 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.295444 (+ 15333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.296930 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.297067 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:42.297396 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:42.298057 (+ 661us) server_negotiation.cc:299] Negotiation successful
0926 15:05:42.298229 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16890,"thread_start_us":16755,"threads_started":1}
I20260926 15:05:42.299365 23116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:42.243935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:34289 (local address 127.0.0.1:43902)
0926 15:05:42.262841 (+ 18906us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:42.262908 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:42.263152 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:42.265739 (+ 2587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:42.265744 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:42.265779 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:05:42.278340 (+ 12561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:42.278357 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.281397 (+ 3040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.281403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:42.287056 (+ 5653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:42.287074 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.297631 (+ 10557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.297638 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:42.297827 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:42.297863 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 15:05:42.298832 (+ 969us) client_negotiation.cc:240] Negotiation successful
0926 15:05:42.298991 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18563,"mutex_wait_us":3648,"thread_start_us":20009,"threads_started":1}
I20260926 15:05:42.303648 22899 tablet_service.cc:1467] Processing CreateTablet for tablet 7d386b307bba4fb9bbe4153fdc521575 (DEFAULT_TABLE table=test-table [id=e74177d8cd114731966c00d7cc7c1871]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:05:42.305226 22899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d386b307bba4fb9bbe4153fdc521575. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:05:42.306500 23119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:42.265964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:40743 (local address 127.0.0.1:60072)
0926 15:05:42.278785 (+ 12821us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:42.278804 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:42.278932 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:42.282054 (+ 3122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:42.282059 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:42.282092 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:05:42.282485 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:42.282498 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.289223 (+ 6725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.289228 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:42.298524 (+ 9296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:42.298545 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.305641 (+ 7096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.305649 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:42.305864 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:42.305914 (+ 50us) client_negotiation.cc:769] Sending connection context
0926 15:05:42.306017 (+ 103us) client_negotiation.cc:240] Negotiation successful
0926 15:05:42.306171 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12677,"thread_start_us":19845,"threads_started":1}
I20260926 15:05:42.306777 23118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:42.264840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60072 (local address 127.9.123.67:40743)
0926 15:05:42.266549 (+ 1709us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:42.266559 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:42.280302 (+ 13743us) server_negotiation.cc:407] Connection header received
0926 15:05:42.280448 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:42.280456 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:42.280791 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:42.281034 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:05:42.287286 (+ 6252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.288998 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.298932 (+ 9934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.300415 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.300549 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:42.300778 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:42.306164 (+ 5386us) server_negotiation.cc:299] Negotiation successful
0926 15:05:42.306377 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1557,"thread_start_us":1412,"threads_started":1}
I20260926 15:05:42.311090 23033 tablet_service.cc:1467] Processing CreateTablet for tablet 7d386b307bba4fb9bbe4153fdc521575 (DEFAULT_TABLE table=test-table [id=e74177d8cd114731966c00d7cc7c1871]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:05:42.313046 23033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d386b307bba4fb9bbe4153fdc521575. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:05:42.325222 23120 tablet_bootstrap.cc:492] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249: Bootstrap starting.
I20260926 15:05:42.329003 23120 tablet_bootstrap.cc:654] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249: Neither blocks nor log segments found. Creating new log.
I20260926 15:05:42.332448 23120 log.cc:824] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249: Log is configured to *not* fsync() on all Append() calls
I20260926 15:05:42.345829 23122 tablet_bootstrap.cc:492] T 7d386b307bba4fb9bbe4153fdc521575 P f400a972edb94463a2b6ec56af27d06d: Bootstrap starting.
I20260926 15:05:42.352000 23122 tablet_bootstrap.cc:654] T 7d386b307bba4fb9bbe4153fdc521575 P f400a972edb94463a2b6ec56af27d06d: Neither blocks nor log segments found. Creating new log.
I20260926 15:05:42.355764 23120 tablet_bootstrap.cc:492] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249: No bootstrap required, opened a new log
I20260926 15:05:42.355947 23122 log.cc:824] T 7d386b307bba4fb9bbe4153fdc521575 P f400a972edb94463a2b6ec56af27d06d: Log is configured to *not* fsync() on all Append() calls
I20260926 15:05:42.356244 23120 ts_tablet_manager.cc:1397] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249: Time spent bootstrapping tablet: real 0.031s user 0.027s sys 0.000s
I20260926 15:05:42.358615 23123 tablet_bootstrap.cc:492] T 7d386b307bba4fb9bbe4153fdc521575 P 503b1ffe2f3c415fbabfb9069bce5854: Bootstrap starting.
I20260926 15:05:42.362419 23123 tablet_bootstrap.cc:654] T 7d386b307bba4fb9bbe4153fdc521575 P 503b1ffe2f3c415fbabfb9069bce5854: Neither blocks nor log segments found. Creating new log.
I20260926 15:05:42.363755 23123 log.cc:824] T 7d386b307bba4fb9bbe4153fdc521575 P 503b1ffe2f3c415fbabfb9069bce5854: Log is configured to *not* fsync() on all Append() calls
I20260926 15:05:42.364492 23122 tablet_bootstrap.cc:492] T 7d386b307bba4fb9bbe4153fdc521575 P f400a972edb94463a2b6ec56af27d06d: No bootstrap required, opened a new log
I20260926 15:05:42.364806 23122 ts_tablet_manager.cc:1397] T 7d386b307bba4fb9bbe4153fdc521575 P f400a972edb94463a2b6ec56af27d06d: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.003s
I20260926 15:05:42.367971 23120 raft_consensus.cc:348] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503b1ffe2f3c415fbabfb9069bce5854" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34289 } } peers { permanent_uuid: "2addb331c3b74e0faa92ca2274455249" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46163 } } peers { permanent_uuid: "f400a972edb94463a2b6ec56af27d06d" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40743 } }
I20260926 15:05:42.368631 23120 raft_consensus.cc:374] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:05:42.368821 23120 raft_consensus.cc:729] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2addb331c3b74e0faa92ca2274455249, State: Initialized, Role: FOLLOWER
I20260926 15:05:42.369735 23120 consensus_queue.cc:260] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503b1ffe2f3c415fbabfb9069bce5854" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34289 } } peers { permanent_uuid: "2addb331c3b74e0faa92ca2274455249" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46163 } } peers { permanent_uuid: "f400a972edb94463a2b6ec56af27d06d" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40743 } }
I20260926 15:05:42.376168 23122 raft_consensus.cc:348] T 7d386b307bba4fb9bbe4153fdc521575 P f400a972edb94463a2b6ec56af27d06d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503b1ffe2f3c415fbabfb9069bce5854" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34289 } } peers { permanent_uuid: "2addb331c3b74e0faa92ca2274455249" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46163 } } peers { permanent_uuid: "f400a972edb94463a2b6ec56af27d06d" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40743 } }
I20260926 15:05:42.376720 23122 raft_consensus.cc:374] T 7d386b307bba4fb9bbe4153fdc521575 P f400a972edb94463a2b6ec56af27d06d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:05:42.376840 23122 raft_consensus.cc:729] T 7d386b307bba4fb9bbe4153fdc521575 P f400a972edb94463a2b6ec56af27d06d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f400a972edb94463a2b6ec56af27d06d, State: Initialized, Role: FOLLOWER
I20260926 15:05:42.377763 23122 consensus_queue.cc:260] T 7d386b307bba4fb9bbe4153fdc521575 P f400a972edb94463a2b6ec56af27d06d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503b1ffe2f3c415fbabfb9069bce5854" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34289 } } peers { permanent_uuid: "2addb331c3b74e0faa92ca2274455249" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46163 } } peers { permanent_uuid: "f400a972edb94463a2b6ec56af27d06d" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40743 } }
I20260926 15:05:42.378614 23120 ts_tablet_manager.cc:1428] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249: Time spent starting tablet: real 0.022s user 0.020s sys 0.000s
I20260926 15:05:42.379207 22820 heartbeater.cc:503] Master 127.9.123.126:45261 was elected leader, sending a full tablet report...
I20260926 15:05:42.387869 23098 heartbeater.cc:503] Master 127.9.123.126:45261 was elected leader, sending a full tablet report...
I20260926 15:05:42.388721 23122 ts_tablet_manager.cc:1428] T 7d386b307bba4fb9bbe4153fdc521575 P f400a972edb94463a2b6ec56af27d06d: Time spent starting tablet: real 0.024s user 0.016s sys 0.008s
W20260926 15:05:42.390039 23099 tablet.cc:2369] T 7d386b307bba4fb9bbe4153fdc521575 P f400a972edb94463a2b6ec56af27d06d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:05:42.394906 23123 tablet_bootstrap.cc:492] T 7d386b307bba4fb9bbe4153fdc521575 P 503b1ffe2f3c415fbabfb9069bce5854: No bootstrap required, opened a new log
I20260926 15:05:42.395241 23123 ts_tablet_manager.cc:1397] T 7d386b307bba4fb9bbe4153fdc521575 P 503b1ffe2f3c415fbabfb9069bce5854: Time spent bootstrapping tablet: real 0.038s user 0.009s sys 0.005s
I20260926 15:05:42.406481 23123 raft_consensus.cc:348] T 7d386b307bba4fb9bbe4153fdc521575 P 503b1ffe2f3c415fbabfb9069bce5854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503b1ffe2f3c415fbabfb9069bce5854" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34289 } } peers { permanent_uuid: "2addb331c3b74e0faa92ca2274455249" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46163 } } peers { permanent_uuid: "f400a972edb94463a2b6ec56af27d06d" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40743 } }
I20260926 15:05:42.407050 23123 raft_consensus.cc:374] T 7d386b307bba4fb9bbe4153fdc521575 P 503b1ffe2f3c415fbabfb9069bce5854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:05:42.407171 23123 raft_consensus.cc:729] T 7d386b307bba4fb9bbe4153fdc521575 P 503b1ffe2f3c415fbabfb9069bce5854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 503b1ffe2f3c415fbabfb9069bce5854, State: Initialized, Role: FOLLOWER
I20260926 15:05:42.408272 23123 consensus_queue.cc:260] T 7d386b307bba4fb9bbe4153fdc521575 P 503b1ffe2f3c415fbabfb9069bce5854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503b1ffe2f3c415fbabfb9069bce5854" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34289 } } peers { permanent_uuid: "2addb331c3b74e0faa92ca2274455249" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46163 } } peers { permanent_uuid: "f400a972edb94463a2b6ec56af27d06d" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40743 } }
I20260926 15:05:42.410130 22965 heartbeater.cc:503] Master 127.9.123.126:45261 was elected leader, sending a full tablet report...
I20260926 15:05:42.410866 23123 ts_tablet_manager.cc:1428] T 7d386b307bba4fb9bbe4153fdc521575 P 503b1ffe2f3c415fbabfb9069bce5854: Time spent starting tablet: real 0.015s user 0.012s sys 0.003s
W20260926 15:05:42.485196 22966 tablet.cc:2369] T 7d386b307bba4fb9bbe4153fdc521575 P 503b1ffe2f3c415fbabfb9069bce5854: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 15:05:42.505204 22822 tablet.cc:2369] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:05:42.570922 23126 raft_consensus.cc:482] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:05:42.571202 23126 raft_consensus.cc:504] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503b1ffe2f3c415fbabfb9069bce5854" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34289 } } peers { permanent_uuid: "2addb331c3b74e0faa92ca2274455249" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46163 } } peers { permanent_uuid: "f400a972edb94463a2b6ec56af27d06d" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40743 } }
I20260926 15:05:42.572933 23126 leader_election.cc:288] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 503b1ffe2f3c415fbabfb9069bce5854 (127.9.123.66:34289), f400a972edb94463a2b6ec56af27d06d (127.9.123.67:40743)
I20260926 15:05:42.578976 23115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:42.573401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:42805 (local address 127.9.123.66:34289)
0926 15:05:42.573634 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:42.573640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:42.573943 (+ 303us) server_negotiation.cc:407] Connection header received
0926 15:05:42.574123 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:42.574129 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:42.574206 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:42.574292 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:05:42.574822 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.575517 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.577282 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.578142 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.578278 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:42.578360 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:42.578610 (+ 250us) server_negotiation.cc:299] Negotiation successful
0926 15:05:42.578730 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 15:05:42.579061 23136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:42.573217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:34289 (local address 127.9.123.65:42805)
0926 15:05:42.573817 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:42.573833 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:42.573992 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:42.574332 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:42.574337 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:42.574374 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:05:42.574684 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:42.574693 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.575638 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.575642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:42.577138 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:42.577153 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.578328 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.578334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:42.578472 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:42.578495 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 15:05:42.578574 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 15:05:42.578708 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":298,"threads_started":1}
I20260926 15:05:42.579416 23118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:42.574061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:60915 (local address 127.9.123.67:40743)
0926 15:05:42.574235 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:42.574241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:42.574611 (+ 370us) server_negotiation.cc:407] Connection header received
0926 15:05:42.574771 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:42.574777 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:42.574836 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:42.574913 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:05:42.575395 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.576063 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.577762 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.578568 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.578741 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:42.578813 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:42.579067 (+ 254us) server_negotiation.cc:299] Negotiation successful
0926 15:05:42.579192 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 15:05:42.579476 23137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:42.573951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:40743 (local address 127.9.123.65:60915)
0926 15:05:42.574508 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:42.574528 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:42.574643 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:42.574934 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:42.574938 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:42.574960 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:05:42.575256 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:42.575264 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.576195 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.576200 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:42.577617 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:42.577629 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.578787 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.578793 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:42.578937 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:42.578958 (+ 21us) client_negotiation.cc:769] Sending connection context
0926 15:05:42.579035 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 15:05:42.579162 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":253,"threads_started":1}
I20260926 15:05:42.580070 22919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d386b307bba4fb9bbe4153fdc521575" candidate_uuid: "2addb331c3b74e0faa92ca2274455249" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "503b1ffe2f3c415fbabfb9069bce5854" is_pre_election: true
I20260926 15:05:42.580556 22919 raft_consensus.cc:2454] T 7d386b307bba4fb9bbe4153fdc521575 P 503b1ffe2f3c415fbabfb9069bce5854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2addb331c3b74e0faa92ca2274455249 in term 0.
I20260926 15:05:42.580528 23053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d386b307bba4fb9bbe4153fdc521575" candidate_uuid: "2addb331c3b74e0faa92ca2274455249" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f400a972edb94463a2b6ec56af27d06d" is_pre_election: true
I20260926 15:05:42.580935 23053 raft_consensus.cc:2454] T 7d386b307bba4fb9bbe4153fdc521575 P f400a972edb94463a2b6ec56af27d06d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2addb331c3b74e0faa92ca2274455249 in term 0.
I20260926 15:05:42.581247 22709 leader_election.cc:302] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2addb331c3b74e0faa92ca2274455249, 503b1ffe2f3c415fbabfb9069bce5854; no voters:
I20260926 15:05:42.581647 23126 raft_consensus.cc:2781] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:05:42.581782 23126 raft_consensus.cc:482] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:05:42.581884 23126 raft_consensus.cc:3037] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:05:42.583982 23126 raft_consensus.cc:504] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503b1ffe2f3c415fbabfb9069bce5854" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34289 } } peers { permanent_uuid: "2addb331c3b74e0faa92ca2274455249" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46163 } } peers { permanent_uuid: "f400a972edb94463a2b6ec56af27d06d" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40743 } }
I20260926 15:05:42.584543 23126 leader_election.cc:288] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249 [CANDIDATE]: Term 1 election: Requested vote from peers 503b1ffe2f3c415fbabfb9069bce5854 (127.9.123.66:34289), f400a972edb94463a2b6ec56af27d06d (127.9.123.67:40743)
I20260926 15:05:42.585065 22919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d386b307bba4fb9bbe4153fdc521575" candidate_uuid: "2addb331c3b74e0faa92ca2274455249" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "503b1ffe2f3c415fbabfb9069bce5854"
I20260926 15:05:42.585192 23053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d386b307bba4fb9bbe4153fdc521575" candidate_uuid: "2addb331c3b74e0faa92ca2274455249" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f400a972edb94463a2b6ec56af27d06d"
I20260926 15:05:42.585299 22919 raft_consensus.cc:3037] T 7d386b307bba4fb9bbe4153fdc521575 P 503b1ffe2f3c415fbabfb9069bce5854 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:05:42.585399 23053 raft_consensus.cc:3037] T 7d386b307bba4fb9bbe4153fdc521575 P f400a972edb94463a2b6ec56af27d06d [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:05:42.587517 22919 raft_consensus.cc:2454] T 7d386b307bba4fb9bbe4153fdc521575 P 503b1ffe2f3c415fbabfb9069bce5854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2addb331c3b74e0faa92ca2274455249 in term 1.
I20260926 15:05:42.587602 23053 raft_consensus.cc:2454] T 7d386b307bba4fb9bbe4153fdc521575 P f400a972edb94463a2b6ec56af27d06d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2addb331c3b74e0faa92ca2274455249 in term 1.
I20260926 15:05:42.587961 22709 leader_election.cc:302] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2addb331c3b74e0faa92ca2274455249, 503b1ffe2f3c415fbabfb9069bce5854; no voters:
I20260926 15:05:42.588230 23126 raft_consensus.cc:2781] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:05:42.588691 23126 raft_consensus.cc:686] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249 [term 1 LEADER]: Becoming Leader. State: Replica: 2addb331c3b74e0faa92ca2274455249, State: Running, Role: LEADER
I20260926 15:05:42.589186 23126 consensus_queue.cc:237] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503b1ffe2f3c415fbabfb9069bce5854" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34289 } } peers { permanent_uuid: "2addb331c3b74e0faa92ca2274455249" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46163 } } peers { permanent_uuid: "f400a972edb94463a2b6ec56af27d06d" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40743 } }
I20260926 15:05:42.593637 22623 catalog_manager.cc:5495] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2addb331c3b74e0faa92ca2274455249 (127.9.123.65). New cstate: current_term: 1 leader_uuid: "2addb331c3b74e0faa92ca2274455249" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "503b1ffe2f3c415fbabfb9069bce5854" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2addb331c3b74e0faa92ca2274455249" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 46163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f400a972edb94463a2b6ec56af27d06d" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40743 } health_report { overall_health: UNKNOWN } } }
I20260926 15:05:42.633836 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:42.628310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56340 (local address 127.9.123.65:46163)
0926 15:05:42.628534 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:42.628540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:42.628561 (+ 21us) server_negotiation.cc:407] Connection header received
0926 15:05:42.628737 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:42.628741 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:42.628830 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:42.628969 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:05:42.629397 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.630148 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.631566 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.631990 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.632110 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:42.632651 (+ 541us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:05:42.633227 (+ 576us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:05:42.633372 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:42.633475 (+ 103us) server_negotiation.cc:299] Negotiation successful
0926 15:05:42.633565 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 15:05:42.642560 22919 raft_consensus.cc:1261] T 7d386b307bba4fb9bbe4153fdc521575 P 503b1ffe2f3c415fbabfb9069bce5854 [term 1 FOLLOWER]: Refusing update from remote peer 2addb331c3b74e0faa92ca2274455249: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:05:42.642560 23053 raft_consensus.cc:1261] T 7d386b307bba4fb9bbe4153fdc521575 P f400a972edb94463a2b6ec56af27d06d [term 1 FOLLOWER]: Refusing update from remote peer 2addb331c3b74e0faa92ca2274455249: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:05:42.643580 23126 consensus_queue.cc:1035] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249 [LEADER]: Connected to new peer: Peer: permanent_uuid: "503b1ffe2f3c415fbabfb9069bce5854" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 34289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:05:42.643890 23138 consensus_queue.cc:1035] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f400a972edb94463a2b6ec56af27d06d" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:05:42.670739 22623 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:05:42.671020 22623 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:05:42.674889 22623 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4125662113d54580b59da35e5c60321a: Sending DeleteTablet for 3 replicas of tablet 7d386b307bba4fb9bbe4153fdc521575
I20260926 15:05:42.676353 22899 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d386b307bba4fb9bbe4153fdc521575 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:05:42 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:43902
I20260926 15:05:42.676712 23033 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d386b307bba4fb9bbe4153fdc521575 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:05:42 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:60072
I20260926 15:05:42.677556 22755 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d386b307bba4fb9bbe4153fdc521575 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:05:42 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:56324
I20260926 15:05:42.678198 23152 tablet_replica.cc:330] T 7d386b307bba4fb9bbe4153fdc521575 P f400a972edb94463a2b6ec56af27d06d: stopping tablet replica
I20260926 15:05:42.678530 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 22660
I20260926 15:05:42.678761 23152 raft_consensus.cc:2229] T 7d386b307bba4fb9bbe4153fdc521575 P f400a972edb94463a2b6ec56af27d06d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:05:42.679315 23152 raft_consensus.cc:2258] T 7d386b307bba4fb9bbe4153fdc521575 P f400a972edb94463a2b6ec56af27d06d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:05:42.681857 23152 ts_tablet_manager.cc:1905] T 7d386b307bba4fb9bbe4153fdc521575 P f400a972edb94463a2b6ec56af27d06d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:05:42.689951 23152 ts_tablet_manager.cc:1918] T 7d386b307bba4fb9bbe4153fdc521575 P f400a972edb94463a2b6ec56af27d06d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:05:42.690178 23152 log.cc:1196] T 7d386b307bba4fb9bbe4153fdc521575 P f400a972edb94463a2b6ec56af27d06d: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/7d386b307bba4fb9bbe4153fdc521575
I20260926 15:05:42.690704 23152 ts_tablet_manager.cc:1939] T 7d386b307bba4fb9bbe4153fdc521575 P f400a972edb94463a2b6ec56af27d06d: Deleting consensus metadata
I20260926 15:05:42.692829 22608 catalog_manager.cc:4841] TS f400a972edb94463a2b6ec56af27d06d (127.9.123.67:40743): tablet 7d386b307bba4fb9bbe4153fdc521575 (table test-table [id=e74177d8cd114731966c00d7cc7c1871]) successfully deleted
I20260926 15:05:42.694507 23151 tablet_replica.cc:330] T 7d386b307bba4fb9bbe4153fdc521575 P 503b1ffe2f3c415fbabfb9069bce5854: stopping tablet replica
I20260926 15:05:42.694916 23151 raft_consensus.cc:2229] T 7d386b307bba4fb9bbe4153fdc521575 P 503b1ffe2f3c415fbabfb9069bce5854 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:05:42.695382 23151 raft_consensus.cc:2258] T 7d386b307bba4fb9bbe4153fdc521575 P 503b1ffe2f3c415fbabfb9069bce5854 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:05:42.697631 23151 ts_tablet_manager.cc:1905] T 7d386b307bba4fb9bbe4153fdc521575 P 503b1ffe2f3c415fbabfb9069bce5854: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:05:42.699623 23153 tablet_replica.cc:330] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249: stopping tablet replica
I20260926 15:05:42.700152 23153 raft_consensus.cc:2229] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249 [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:05:42.700899 23153 raft_consensus.cc:2258] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:05:42.704219 23151 ts_tablet_manager.cc:1918] T 7d386b307bba4fb9bbe4153fdc521575 P 503b1ffe2f3c415fbabfb9069bce5854: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:05:42.704411 23151 log.cc:1196] T 7d386b307bba4fb9bbe4153fdc521575 P 503b1ffe2f3c415fbabfb9069bce5854: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/7d386b307bba4fb9bbe4153fdc521575
I20260926 15:05:42.705263 23151 ts_tablet_manager.cc:1939] T 7d386b307bba4fb9bbe4153fdc521575 P 503b1ffe2f3c415fbabfb9069bce5854: Deleting consensus metadata
I20260926 15:05:42.705473 23117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:42.680736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56346 (local address 127.9.123.65:46163)
0926 15:05:42.680932 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:42.680938 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:42.685818 (+ 4880us) server_negotiation.cc:407] Connection header received
0926 15:05:42.686024 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:42.686030 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:42.686127 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:42.686244 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:42.688528 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.689849 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.691994 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.692683 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.692816 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:42.696878 (+ 4062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:42.696894 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:42.696913 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:42.696968 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:42.701477 (+ 4509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:42.702850 (+ 1373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:42.702860 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:42.702863 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:42.703066 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:42.703531 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:42.703537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:42.703541 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:42.704001 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:42.704418 (+ 417us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:42.704613 (+ 195us) server_negotiation.cc:299] Negotiation successful
0926 15:05:42.704981 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 15:05:42.706094 23153 ts_tablet_manager.cc:1905] T 7d386b307bba4fb9bbe4153fdc521575 P 2addb331c3b74e0faa92ca2274455249: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:05:42.706489 22815 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:05:42.707019 22608 catalog_manager.cc:4841] TS 503b1ffe2f3c415fbabfb9069bce5854 (127.9.123.66:34289): tablet 7d386b307bba4fb9bbe4153fdc521575 (table test-table [id=e74177d8cd114731966c00d7cc7c1871]) successfully deleted
I20260926 15:05:42.809441 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 22660
W20260926 15:05:42.823628 22609 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.123.65:46163 (error 108)
I20260926 15:05:42.824303 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 22821
I20260926 15:05:42.825410 23116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:42.824879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:46163 (local address 127.0.0.1:56348)
0926 15:05:42.825068 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:42.825227 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:46163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260926 15:05:42.825629 22609 catalog_manager.cc:4584] TS 2addb331c3b74e0faa92ca2274455249 (127.9.123.65:46163): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7d386b307bba4fb9bbe4153fdc521575: Network error: Client connection negotiation failed: client connection to 127.9.123.65:46163: connect: Connection refused (error 111)
I20260926 15:05:42.838135 23115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:42.825048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43912 (local address 127.9.123.66:34289)
0926 15:05:42.825232 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:42.825237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:42.825291 (+ 54us) server_negotiation.cc:407] Connection header received
0926 15:05:42.825559 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:42.825565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:42.825651 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:42.825745 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:42.827236 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.827977 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.829361 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.829774 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.829873 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:42.832477 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:42.832492 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:42.832509 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:42.832547 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:42.835453 (+ 2906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:42.836246 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:42.836256 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:42.836259 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:42.836406 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:42.836751 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:42.836754 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:42.836756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:42.837163 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:42.837397 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:42.837618 (+ 221us) server_negotiation.cc:299] Negotiation successful
0926 15:05:42.837807 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260926 15:05:42.838794 22960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:05:42.879571 23116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:42.878969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:46163 (local address 127.0.0.1:56354)
0926 15:05:42.879234 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:42.879360 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:46163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260926 15:05:42.880740 23116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:42.880315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:46163 (local address 127.0.0.1:56358)
0926 15:05:42.880485 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:42.880594 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:46163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260926 15:05:42.929726 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 22821
I20260926 15:05:42.941579 23116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:42.940960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:46163 (local address 127.0.0.1:56364)
0926 15:05:42.941243 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:42.941353 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:46163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260926 15:05:42.942780 23116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:42.942379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:46163 (local address 127.0.0.1:56374)
0926 15:05:42.942547 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:42.942633 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:46163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260926 15:05:42.943220 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 22969
I20260926 15:05:42.956537 23118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:42.943944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60084 (local address 127.9.123.67:40743)
0926 15:05:42.944116 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:42.944121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:42.944140 (+ 19us) server_negotiation.cc:407] Connection header received
0926 15:05:42.944245 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:42.944249 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:42.944311 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:42.944398 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:42.945821 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.946614 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.947998 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:42.948405 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:42.948517 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:42.951146 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:42.951162 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:42.951179 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:42.951233 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:42.954023 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:42.954735 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:42.954742 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:42.954744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:42.954872 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:42.955166 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:42.955169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:42.955171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:42.955533 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:42.955747 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:42.956059 (+ 312us) server_negotiation.cc:299] Negotiation successful
0926 15:05:42.956246 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 15:05:42.957381 23093 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:05:43.023577 23116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:43.022974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:46163 (local address 127.0.0.1:56376)
0926 15:05:43.023235 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:43.023359 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:46163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260926 15:05:43.024905 23116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:43.024407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:46163 (local address 127.0.0.1:56378)
0926 15:05:43.024649 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:43.024746 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:46163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260926 15:05:43.046432 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 22969
I20260926 15:05:43.059470 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 22590
I20260926 15:05:43.060709 22653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:05:43.174031 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 22590
I20260926 15:05:43.192471 9709 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260926 15:05:43.528650 9709 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260926 15:05:43.528924 9709 mini_ranger.cc:66] Stopping Ranger...
I20260926 15:05:43.865057 9709 mini_ranger.cc:68] Stopped Ranger
2026-09-26 15:05:43.865 UTC [21642] LOG: received smart shutdown request
2026-09-26 15:05:43.869 UTC [21642] LOG: background worker "logical replication launcher" (PID 21651) exited with exit code 1
2026-09-26 15:05:43.869 UTC [21646] LOG: shutting down
2026-09-26 15:05:44.078 UTC [21642] LOG: database system is shut down
2026-09-26T15:05:44Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (54591 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:05:44 dist-test-slave-7153 krb5kdc[23168](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:05:44 dist-test-slave-7153 krb5kdc[23168](info): set up 2 sockets
Sep 26 15:05:44 dist-test-slave-7153 krb5kdc[23168](info): commencing operation
krb5kdc: starting...
W20260926 15:05:45.229543 9709 mini_kdc.cc:121] Time spent starting KDC: real 1.002s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:05:45 dist-test-slave-7153 krb5kdc[23168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:05:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:05:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:05:45.466053 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:39219
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:45221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:39219,127.9.123.125:37155,127.9.123.124:39045
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:05:45.811441 23181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:05:45.811791 23181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:05:45.811928 23181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:05:45.819433 23181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:05:45.819581 23181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:05:45.819653 23181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:05:45.819734 23181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:05:45.819795 23181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:05:45.830127 23181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:45221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:39219,127.9.123.125:37155,127.9.123.124:39045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:39219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:05:45.831610 23181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:05:45.833014 23181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:05:45.845126 23187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:45.845137 23189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:45.845088 23186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:45.846297 23181 server_base.cc:1034] running on GCE node
I20260926 15:05:45.847162 23181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:05:45.848721 23181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:05:45.849953 23181 hybrid_clock.cc:648] HybridClock initialized: now 1790435145849896 us; error 54 us; skew 500 ppm
I20260926 15:05:45.850346 23181 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:05:45 dist-test-slave-7153 krb5kdc[23168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435145, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:45.858404 23181 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:05:45.860286 23181 webserver.cc:460] Webserver started at http://127.9.123.126:36259/ using document root <none> and password file <none>
I20260926 15:05:45.861061 23181 fs_manager.cc:359] Metadata directory not provided
I20260926 15:05:45.861241 23181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:05:45.861568 23181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:05:45.864746 23181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "eee7c64a76464fae99aafec5a4fe2c5f"
format_stamp: "Formatted at 2026-09-26 15:05:45 on dist-test-slave-7153"
server_key: "0fc63f216ac9ee37c1f29e108cd91739"
server_key_iv: "934dfcafa07db0c07b2ba7a28e5db5be"
server_key_version: "encryptionkey@0"
I20260926 15:05:45.865648 23181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "eee7c64a76464fae99aafec5a4fe2c5f"
format_stamp: "Formatted at 2026-09-26 15:05:45 on dist-test-slave-7153"
server_key: "0fc63f216ac9ee37c1f29e108cd91739"
server_key_iv: "934dfcafa07db0c07b2ba7a28e5db5be"
server_key_version: "encryptionkey@0"
I20260926 15:05:45.871557 23181 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 15:05:45.875913 23196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:45.877556 23181 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 15:05:45.877760 23181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "eee7c64a76464fae99aafec5a4fe2c5f"
format_stamp: "Formatted at 2026-09-26 15:05:45 on dist-test-slave-7153"
server_key: "0fc63f216ac9ee37c1f29e108cd91739"
server_key_iv: "934dfcafa07db0c07b2ba7a28e5db5be"
server_key_version: "encryptionkey@0"
I20260926 15:05:45.878024 23181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:05:45.895054 23181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:05:45.900692 23181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:05:45.901024 23181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:05:45.921296 23181 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:39219
I20260926 15:05:45.921314 23247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:39219 every 8 connection(s)
I20260926 15:05:45.922832 23181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:05:45.928707 23248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:05:45.929589 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 23181
I20260926 15:05:45.929936 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:05:45.930644 9709 external_mini_cluster.cc:1428] Setting key 25ec150b40e3c41debd8b43aa6f33d13
I20260926 15:05:45.938747 23248 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39219 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.123.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.125@KRBTEST.COM" created.
Sep 26 15:05:45 dist-test-slave-7153 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435145, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:05:45.967656 23252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:45.944161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55474 (local address 127.9.123.126:39219)
0926 15:05:45.946380 (+ 2219us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:45.946388 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:45.946434 (+ 46us) server_negotiation.cc:407] Connection header received
0926 15:05:45.948493 (+ 2059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:45.948500 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:45.948875 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:45.949084 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:45.950934 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:45.952301 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:45.954533 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:45.955257 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:45.955454 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:45.959864 (+ 4410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:45.959874 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:45.959889 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:45.959924 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:45.962899 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:45.963926 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:45.963936 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:45.963939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:45.964211 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:45.964709 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:45.964714 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:45.964718 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:45.965452 (+ 734us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:45.965659 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:45.966268 (+ 609us) server_negotiation.cc:299] Negotiation successful
0926 15:05:45.966709 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":136,"server-negotiator.queue_time_us":1978,"spinlock_wait_cycles":17152,"thread_start_us":1717,"threads_started":1}
I20260926 15:05:45.967687 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:45.944161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39219 (local address 127.0.0.1:55474)
0926 15:05:45.945219 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:45.945275 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:45.947976 (+ 2701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:45.949204 (+ 1228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:45.949217 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:45.950097 (+ 880us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:05:45.950657 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:45.950689 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:45.952509 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:45.952516 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:45.954205 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:45.954244 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:45.955649 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:45.955657 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:45.955896 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:45.956778 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:05:45.956806 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:05:45.959519 (+ 2713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:05:45.963088 (+ 3569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:45.963103 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:45.963119 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:45.963761 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:45.964377 (+ 616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:45.964382 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:45.964386 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:45.964591 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:45.965613 (+ 1022us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:05:45.965620 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:05:45.965841 (+ 221us) client_negotiation.cc:769] Sending connection context
0926 15:05:45.966077 (+ 236us) client_negotiation.cc:240] Negotiation successful
0926 15:05:45.966518 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":436,"threads_started":1}
I20260926 15:05:45.971192 23248 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:05:45.972363 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:45.971904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:37155 (local address 127.0.0.1:34474)
0926 15:05:45.972107 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:45.972225 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:37155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260926 15:05:45.972642 23200 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.123.125:37155: connect: Connection refused (error 111)
I20260926 15:05:45.974241 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:45.973890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:37155 (local address 127.0.0.1:34490)
0926 15:05:45.974034 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:45.974105 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:37155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260926 15:05:45.974453 23248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.125:37155: Network error: Client connection negotiation failed: client connection to 127.9.123.125:37155: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.125.keytab.
Entry for principal kudu/127.9.123.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.125.keytab.
I20260926 15:05:45.986343 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.125:37155
--webserver_interface=127.9.123.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.125.keytab
--principal=kudu/127.9.123.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:45221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:39219,127.9.123.125:37155,127.9.123.124:39045
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:05:46.023823 23248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } attempt: 1
I20260926 15:05:46.025307 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.024580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:37155 (local address 127.0.0.1:34498)
0926 15:05:46.024829 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.024969 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:37155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260926 15:05:46.026702 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.026230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:37155 (local address 127.0.0.1:34508)
0926 15:05:46.026395 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.026510 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:37155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260926 15:05:46.026947 23248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.125:37155: Network error: Client connection negotiation failed: client connection to 127.9.123.125:37155: connect: Connection refused (error 111)
I20260926 15:05:46.095247 23248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } attempt: 2
I20260926 15:05:46.096695 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.096076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:37155 (local address 127.0.0.1:34510)
0926 15:05:46.096370 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.096506 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:37155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260926 15:05:46.097914 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.097479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:37155 (local address 127.0.0.1:34518)
0926 15:05:46.097632 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.097738 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:37155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 15:05:46.098153 23248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.125:37155: Network error: Client connection negotiation failed: client connection to 127.9.123.125:37155: connect: Connection refused (error 111)
I20260926 15:05:46.189459 23248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } attempt: 3
I20260926 15:05:46.190814 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.190241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:37155 (local address 127.0.0.1:34528)
0926 15:05:46.190480 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.190640 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:37155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260926 15:05:46.191959 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.191537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:37155 (local address 127.0.0.1:34544)
0926 15:05:46.191691 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.191793 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:37155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 15:05:46.192188 23248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.125:37155: Network error: Client connection negotiation failed: client connection to 127.9.123.125:37155: connect: Connection refused (error 111)
W20260926 15:05:46.325881 23255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:05:46.326218 23255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:05:46.326347 23255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:05:46.334035 23255 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:05:46.334164 23255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:05:46.334218 23255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:05:46.334295 23255 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:05:46.334357 23255 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:05:46.335423 23248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } attempt: 4
I20260926 15:05:46.336709 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.336109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:37155 (local address 127.0.0.1:34556)
0926 15:05:46.336407 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.336534 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:37155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260926 15:05:46.337872 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.337450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:37155 (local address 127.0.0.1:34572)
0926 15:05:46.337606 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.337718 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:37155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 15:05:46.338088 23248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.125:37155: Network error: Client connection negotiation failed: client connection to 127.9.123.125:37155: connect: Connection refused (error 111)
I20260926 15:05:46.345738 23255 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:45221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:39219,127.9.123.125:37155,127.9.123.124:39045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.125.keytab
--principal=kudu/127.9.123.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.125:37155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.123.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:05:46.347139 23255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:05:46.348501 23255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:05:46.360709 23260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:46.360736 23263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:46.361024 23261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:46.362030 23255 server_base.cc:1034] running on GCE node
I20260926 15:05:46.362875 23255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:05:46.364357 23255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:05:46.365594 23255 hybrid_clock.cc:648] HybridClock initialized: now 1790435146365546 us; error 39 us; skew 500 ppm
I20260926 15:05:46.366036 23255 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:05:46 dist-test-slave-7153 krb5kdc[23168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435146, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:46.369283 23255 init.cc:377] Logged in from keytab as kudu/127.9.123.125@KRBTEST.COM (short username kudu)
I20260926 15:05:46.371037 23255 webserver.cc:460] Webserver started at http://127.9.123.125:44337/ using document root <none> and password file <none>
I20260926 15:05:46.371834 23255 fs_manager.cc:359] Metadata directory not provided
I20260926 15:05:46.371960 23255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:05:46.372272 23255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:05:46.375432 23255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-1/data/instance:
uuid: "efa6b477c5f74291bad537cac10bb128"
format_stamp: "Formatted at 2026-09-26 15:05:46 on dist-test-slave-7153"
server_key: "e6d31094af72c455d573b479af3adafa"
server_key_iv: "21316a29de1f09be99d2e81684e5eb45"
server_key_version: "encryptionkey@0"
I20260926 15:05:46.376264 23255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-1/wal/instance:
uuid: "efa6b477c5f74291bad537cac10bb128"
format_stamp: "Formatted at 2026-09-26 15:05:46 on dist-test-slave-7153"
server_key: "e6d31094af72c455d573b479af3adafa"
server_key_iv: "21316a29de1f09be99d2e81684e5eb45"
server_key_version: "encryptionkey@0"
I20260926 15:05:46.381672 23255 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 15:05:46.385579 23270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:46.387072 23255 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 15:05:46.387249 23255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-1/wal
uuid: "efa6b477c5f74291bad537cac10bb128"
format_stamp: "Formatted at 2026-09-26 15:05:46 on dist-test-slave-7153"
server_key: "e6d31094af72c455d573b479af3adafa"
server_key_iv: "21316a29de1f09be99d2e81684e5eb45"
server_key_version: "encryptionkey@0"
I20260926 15:05:46.387449 23255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:05:46.403908 23255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:05:46.408574 23255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:05:46.408870 23255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:05:46.427685 23255 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.125:37155
I20260926 15:05:46.427718 23321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.125:37155 every 8 connection(s)
I20260926 15:05:46.429214 23255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-1/data/info.pb
I20260926 15:05:46.433359 23322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:05:46.438946 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 23255
I20260926 15:05:46.439160 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-1/wal/instance
I20260926 15:05:46.439716 9709 external_mini_cluster.cc:1428] Setting key ccf93abe8558ee7fff599e538510f0d0
I20260926 15:05:46.441458 23322 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39219 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:05:46 dist-test-slave-7153 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435146, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.125@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.123.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.124@KRBTEST.COM" created.
I20260926 15:05:46.465149 23252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.444943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55490 (local address 127.9.123.126:39219)
0926 15:05:46.445193 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:46.445199 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:46.446662 (+ 1463us) server_negotiation.cc:407] Connection header received
0926 15:05:46.448843 (+ 2181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:46.448849 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:46.448940 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:46.449058 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:46.450755 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:46.451721 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:46.453441 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:46.454049 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:46.454174 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:46.458115 (+ 3941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:46.458123 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:46.458128 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:46.458178 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:46.461725 (+ 3547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:46.462707 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:46.462713 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:46.462715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:46.462861 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:46.463358 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:46.463364 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:46.463366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:46.463700 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:46.463832 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:46.464668 (+ 836us) server_negotiation.cc:299] Negotiation successful
0926 15:05:46.464863 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260926 15:05:46.466596 23324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.445575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39219 (local address 127.0.0.1:55490)
0926 15:05:46.446463 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.446519 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:46.448524 (+ 2005us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:46.449313 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:46.449323 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:46.449906 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:05:46.450555 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:46.450576 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:46.451903 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:46.451907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:46.453290 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:46.453309 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:46.454283 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:46.454289 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:46.454507 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:46.455241 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:05:46.455268 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:05:46.457798 (+ 2530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:05:46.461966 (+ 4168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:46.461976 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:46.461997 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:46.462529 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:46.463005 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:46.463009 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:46.463012 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:46.463235 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:46.463825 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:05:46.463832 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:05:46.464240 (+ 408us) client_negotiation.cc:769] Sending connection context
0926 15:05:46.464525 (+ 285us) client_negotiation.cc:240] Negotiation successful
0926 15:05:46.465140 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":341,"threads_started":1}
I20260926 15:05:46.468397 23322 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:05:46 dist-test-slave-7153 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435146, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.125@KRBTEST.COM for kudu/127.9.123.125@KRBTEST.COM
I20260926 15:05:46.483276 23324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.469080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:37155 (local address 127.0.0.1:34582)
0926 15:05:46.469300 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.469317 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:46.469411 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:46.470412 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:46.470416 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:46.470846 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:05:46.470995 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:46.471004 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:46.472338 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:46.472342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:46.473220 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:46.473232 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:46.474034 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:46.474039 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:46.474255 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:46.474850 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:05:46.474869 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:05:46.477157 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:05:46.480463 (+ 3306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:46.480469 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:46.480474 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:46.480924 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:46.481374 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:46.481378 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:46.481382 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:46.481551 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:46.482224 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:05:46.482248 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:05:46.482414 (+ 166us) client_negotiation.cc:769] Sending connection context
0926 15:05:46.482556 (+ 142us) client_negotiation.cc:240] Negotiation successful
0926 15:05:46.482856 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260926 15:05:46.483362 23326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.469362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34582 (local address 127.9.123.125:37155)
0926 15:05:46.469989 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:46.469996 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:46.470021 (+ 25us) server_negotiation.cc:407] Connection header received
0926 15:05:46.470066 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:46.470072 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:46.470267 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:46.470412 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:46.471152 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:46.472208 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:46.473341 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:46.473897 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:46.474010 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:46.477365 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:46.477378 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:46.477393 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:46.477424 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:46.480278 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:46.481073 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:46.481081 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:46.481083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:46.481271 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:46.481666 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:46.481670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:46.481672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:46.482097 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:46.482304 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:46.482728 (+ 424us) server_negotiation.cc:299] Negotiation successful
0926 15:05:46.482980 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":476,"thread_start_us":332,"threads_started":1}
I20260926 15:05:46.485867 23322 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:05:46.487403 23324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.486834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:39045 (local address 127.0.0.1:37124)
0926 15:05:46.487060 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.487218 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260926 15:05:46.487762 23273 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
I20260926 15:05:46.489594 23324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.489159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:39045 (local address 127.0.0.1:37136)
0926 15:05:46.489353 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.489458 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260926 15:05:46.489835 23322 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:39045: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.124.keytab.
Entry for principal kudu/127.9.123.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.124.keytab.
I20260926 15:05:46.498113 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.124:39045
--webserver_interface=127.9.123.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.124.keytab
--principal=kudu/127.9.123.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:45221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:39219,127.9.123.125:37155,127.9.123.124:39045
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:05:46.539201 23322 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } attempt: 1
I20260926 15:05:46.540438 23324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.539950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:39045 (local address 127.0.0.1:37152)
0926 15:05:46.540165 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.540278 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260926 15:05:46.541602 23324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.541221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:39045 (local address 127.0.0.1:37164)
0926 15:05:46.541392 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.541482 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 15:05:46.541828 23322 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:39045: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
I20260926 15:05:46.610095 23322 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } attempt: 2
I20260926 15:05:46.611330 23324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.610835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:39045 (local address 127.0.0.1:37172)
0926 15:05:46.611068 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.611167 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260926 15:05:46.612378 23324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.612006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:39045 (local address 127.0.0.1:37178)
0926 15:05:46.612171 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.612250 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260926 15:05:46.612618 23322 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:39045: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
I20260926 15:05:46.637382 23248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } attempt: 5
Sep 26 15:05:46 dist-test-slave-7153 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435145, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for kudu/127.9.123.125@KRBTEST.COM
I20260926 15:05:46.650936 23326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.638327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34596 (local address 127.9.123.125:37155)
0926 15:05:46.638494 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:46.638499 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:46.638519 (+ 20us) server_negotiation.cc:407] Connection header received
0926 15:05:46.638632 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:46.638636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:46.638697 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:46.638779 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:46.639763 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:46.640512 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:46.641794 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:46.642265 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:46.642392 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:46.645819 (+ 3427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:46.645824 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:46.645828 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:46.645865 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:46.648348 (+ 2483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:46.649041 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:46.649046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:46.649048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:46.649213 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:46.649570 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:46.649575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:46.649579 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:46.649879 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:46.650017 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:46.650410 (+ 393us) server_negotiation.cc:299] Negotiation successful
0926 15:05:46.650626 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 15:05:46.650959 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.638139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:37155 (local address 127.0.0.1:34596)
0926 15:05:46.638344 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.638367 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:46.638496 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:46.638840 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:46.638845 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:46.639344 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:05:46.639576 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:46.639590 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:46.640668 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:46.640673 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:46.641625 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:46.641639 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:46.642422 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:46.642428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:46.642607 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:46.643259 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:05:46.643278 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:05:46.645588 (+ 2310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:05:46.648539 (+ 2951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:46.648545 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:46.648550 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:46.648920 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:46.649318 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:46.649323 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:46.649327 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:46.649470 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:46.649992 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:05:46.649998 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:05:46.650153 (+ 155us) client_negotiation.cc:769] Sending connection context
0926 15:05:46.650278 (+ 125us) client_negotiation.cc:240] Negotiation successful
0926 15:05:46.650608 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260926 15:05:46.652047 23248 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:05:46.653330 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.652897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:39045 (local address 127.0.0.1:37194)
0926 15:05:46.653070 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.653189 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260926 15:05:46.654331 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.654016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:39045 (local address 127.0.0.1:37208)
0926 15:05:46.654150 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.654220 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 15:05:46.654559 23248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:39045: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
I20260926 15:05:46.703858 23322 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } attempt: 3
I20260926 15:05:46.705072 23324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.704581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:39045 (local address 127.0.0.1:37216)
0926 15:05:46.704803 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.704911 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260926 15:05:46.705821 23248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } attempt: 1
I20260926 15:05:46.706480 23324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.705848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:39045 (local address 127.0.0.1:37228)
0926 15:05:46.706216 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.706322 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250}
W20260926 15:05:46.706732 23322 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:39045: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
I20260926 15:05:46.707052 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.706512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:39045 (local address 127.0.0.1:37244)
0926 15:05:46.706774 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.706885 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
I20260926 15:05:46.708154 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.707787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:39045 (local address 127.0.0.1:37254)
0926 15:05:46.707946 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.708042 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 15:05:46.708379 23248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:39045: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
I20260926 15:05:46.776625 23248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } attempt: 2
I20260926 15:05:46.777817 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.777334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:39045 (local address 127.0.0.1:37256)
0926 15:05:46.777553 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.777653 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260926 15:05:46.778810 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.778462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:39045 (local address 127.0.0.1:37272)
0926 15:05:46.778610 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.778682 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 15:05:46.779033 23248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:39045: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
W20260926 15:05:46.840382 23329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:05:46.840716 23329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:05:46.840852 23329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:05:46.848520 23329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:05:46.848655 23329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:05:46.848737 23329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:05:46.848811 23329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:05:46.848871 23329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:05:46.849993 23322 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } attempt: 4
I20260926 15:05:46.851147 23324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.850658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:39045 (local address 127.0.0.1:37274)
0926 15:05:46.850890 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.850985 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260926 15:05:46.852310 23324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.851828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:39045 (local address 127.0.0.1:37290)
0926 15:05:46.851986 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.852153 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 15:05:46.852607 23322 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:39045: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
I20260926 15:05:46.860162 23329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:45221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:39219,127.9.123.125:37155,127.9.123.124:39045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.124.keytab
--principal=kudu/127.9.123.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.124:39045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.123.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:05:46.861660 23329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:05:46.863080 23329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:05:46.876092 23334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:46.876127 23335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:46.876158 23337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:46.877507 23329 server_base.cc:1034] running on GCE node
I20260926 15:05:46.878376 23329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:05:46.879985 23329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:05:46.881201 23329 hybrid_clock.cc:648] HybridClock initialized: now 1790435146881155 us; error 43 us; skew 500 ppm
I20260926 15:05:46.881610 23329 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:05:46 dist-test-slave-7153 krb5kdc[23168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435146, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:46.885001 23329 init.cc:377] Logged in from keytab as kudu/127.9.123.124@KRBTEST.COM (short username kudu)
I20260926 15:05:46.885200 23248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } attempt: 3
I20260926 15:05:46.886313 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.885863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:39045 (local address 127.0.0.1:37300)
0926 15:05:46.886072 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.886160 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260926 15:05:46.886919 23329 webserver.cc:460] Webserver started at http://127.9.123.124:36179/ using document root <none> and password file <none>
I20260926 15:05:46.887419 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.886967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:39045 (local address 127.0.0.1:37304)
0926 15:05:46.887209 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.887303 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260926 15:05:46.887629 23248 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:39045: Network error: Client connection negotiation failed: client connection to 127.9.123.124:39045: connect: Connection refused (error 111)
I20260926 15:05:46.887781 23329 fs_manager.cc:359] Metadata directory not provided
I20260926 15:05:46.887902 23329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:05:46.888193 23329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:05:46.891546 23329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-2/data/instance:
uuid: "5137e855a1954ebea5ad88ebe4a785cb"
format_stamp: "Formatted at 2026-09-26 15:05:46 on dist-test-slave-7153"
server_key: "9fb55cb17e428c23ed902358e631f557"
server_key_iv: "d8a4d442a09c4027a78428e2a69dca59"
server_key_version: "encryptionkey@0"
I20260926 15:05:46.892448 23329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-2/wal/instance:
uuid: "5137e855a1954ebea5ad88ebe4a785cb"
format_stamp: "Formatted at 2026-09-26 15:05:46 on dist-test-slave-7153"
server_key: "9fb55cb17e428c23ed902358e631f557"
server_key_iv: "d8a4d442a09c4027a78428e2a69dca59"
server_key_version: "encryptionkey@0"
I20260926 15:05:46.898120 23329 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 15:05:46.902122 23344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:46.903748 23329 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 15:05:46.903971 23329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-2/wal
uuid: "5137e855a1954ebea5ad88ebe4a785cb"
format_stamp: "Formatted at 2026-09-26 15:05:46 on dist-test-slave-7153"
server_key: "9fb55cb17e428c23ed902358e631f557"
server_key_iv: "d8a4d442a09c4027a78428e2a69dca59"
server_key_version: "encryptionkey@0"
I20260926 15:05:46.904166 23329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:05:46.921530 23329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:05:46.926460 23329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:05:46.926748 23329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:05:46.947098 23329 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.124:39045
I20260926 15:05:46.947108 23395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.124:39045 every 8 connection(s)
I20260926 15:05:46.948663 23329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-2/data/info.pb
I20260926 15:05:46.950750 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 23329
I20260926 15:05:46.950966 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790434939095847-9709-0/minicluster-data/master-2/wal/instance
I20260926 15:05:46.951574 9709 external_mini_cluster.cc:1428] Setting key b59f769b5468a609c7ba0972cc1bdf7d
I20260926 15:05:46.953961 23396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:05:46.964223 23396 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39219 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:05:46 dist-test-slave-7153 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.124@KRBTEST.COM
Sep 26 15:05:46 dist-test-slave-7153 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435146, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.124@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:05:46.979579 23399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.953883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37306 (local address 127.9.123.124:39045)
0926 15:05:46.955168 (+ 1285us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:46.955182 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:46.955228 (+ 46us) server_negotiation.cc:407] Connection header received
0926 15:05:46.957521 (+ 2293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:46.957548 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:46.958045 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:46.958775 (+ 730us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:46.960787 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:46.962566 (+ 1779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:46.964165 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:46.964961 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:46.965209 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:46.969962 (+ 4753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:46.969978 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:46.969994 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:46.970043 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:46.973665 (+ 3622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:46.974608 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:46.974618 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:46.974627 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:46.974880 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:46.975389 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:46.975396 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:46.975399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:46.976390 (+ 991us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:46.976643 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:46.977562 (+ 919us) server_negotiation.cc:299] Negotiation successful
0926 15:05:46.978043 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1041,"thread_start_us":595,"threads_started":1}
I20260926 15:05:46.986696 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.967220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55506 (local address 127.9.123.126:39219)
0926 15:05:46.967787 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:46.967793 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:46.968378 (+ 585us) server_negotiation.cc:407] Connection header received
0926 15:05:46.968592 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:46.968596 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:46.968668 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:46.968760 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:46.970207 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:46.970961 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:46.973074 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:46.973583 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:46.973698 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:46.978236 (+ 4538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:46.978242 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:46.978247 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:46.978292 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:46.983419 (+ 5127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:46.984414 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:46.984423 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:46.984425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:46.984576 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:46.984986 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:46.984990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:46.984992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:46.985318 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:46.985470 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:46.985982 (+ 512us) server_negotiation.cc:299] Negotiation successful
0926 15:05:46.986228 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":269,"threads_started":1}
I20260926 15:05:46.986678 23401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.967198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39219 (local address 127.0.0.1:55506)
0926 15:05:46.968228 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.968276 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:46.968459 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:46.968864 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:46.968872 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:46.969587 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:05:46.970002 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:46.970014 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:46.971145 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:46.971151 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:46.972847 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:46.972867 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:46.973785 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:46.973793 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:46.973949 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:46.974631 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:05:46.974734 (+ 103us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:05:46.977967 (+ 3233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:05:46.983575 (+ 5608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:46.983586 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:46.983591 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:46.984283 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:46.984696 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:46.984702 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:46.984706 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:46.984864 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:46.985449 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:05:46.985458 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:05:46.985663 (+ 205us) client_negotiation.cc:769] Sending connection context
0926 15:05:46.985848 (+ 185us) client_negotiation.cc:240] Negotiation successful
0926 15:05:46.986215 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"spinlock_wait_cycles":26496,"thread_start_us":536,"threads_started":1}
I20260926 15:05:46.988193 23396 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:05:46 dist-test-slave-7153 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
Sep 26 15:05:46 dist-test-slave-7153 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435146, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.124@KRBTEST.COM for kudu/127.9.123.125@KRBTEST.COM
I20260926 15:05:47.002249 23402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.983833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55512 (local address 127.9.123.126:39219)
0926 15:05:46.984441 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:46.984445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:46.984467 (+ 22us) server_negotiation.cc:407] Connection header received
0926 15:05:46.984516 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:46.984519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:46.984584 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:46.984685 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:46.986102 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:46.987170 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:46.988565 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:46.989176 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:46.989299 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:46.994694 (+ 5395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:46.994705 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:46.994711 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:46.994756 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:46.998285 (+ 3529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:46.999304 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:46.999310 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:46.999312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:46.999528 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:47.000073 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:47.000078 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:47.000082 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:47.000505 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:47.000805 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:47.001497 (+ 692us) server_negotiation.cc:299] Negotiation successful
0926 15:05:47.001766 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":271,"threads_started":1}
I20260926 15:05:47.004664 23401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.988863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:37155 (local address 127.0.0.1:34610)
0926 15:05:46.989076 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:46.989134 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:46.989249 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:46.989669 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:46.989673 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:46.990155 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:05:46.990327 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:46.990337 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:46.991828 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:46.991833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:46.992938 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:46.992954 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:46.993871 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:46.993876 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:46.994060 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:46.994739 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:05:46.994761 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:05:46.997511 (+ 2750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:05:47.001378 (+ 3867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:47.001384 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:47.001388 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:47.002021 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:47.002504 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:47.002509 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:47.002512 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:47.002723 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:47.003596 (+ 873us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:05:47.003601 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:05:47.003826 (+ 225us) client_negotiation.cc:769] Sending connection context
0926 15:05:47.004000 (+ 174us) client_negotiation.cc:240] Negotiation successful
0926 15:05:47.004309 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260926 15:05:47.004925 23326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:46.988997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34610 (local address 127.9.123.125:37155)
0926 15:05:46.989182 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:46.989187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:46.989204 (+ 17us) server_negotiation.cc:407] Connection header received
0926 15:05:46.989377 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:46.989389 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:46.989475 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:46.989581 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:46.990560 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:46.991648 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:46.993093 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:46.993672 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:46.993830 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:46.997792 (+ 3962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:46.997798 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:46.997802 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:46.997848 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:47.001220 (+ 3372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:47.002188 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:47.002193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:47.002195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:47.002343 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:47.002944 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:47.002951 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:47.002954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:47.003338 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:47.003512 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:47.004174 (+ 662us) server_negotiation.cc:299] Negotiation successful
0926 15:05:47.004538 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260926 15:05:47.009203 23396 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:05:47 dist-test-slave-7153 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435146, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.124@KRBTEST.COM for kudu/127.9.123.124@KRBTEST.COM
Sep 26 15:05:47 dist-test-slave-7153 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.125@KRBTEST.COM
I20260926 15:05:47.023319 23399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:47.010047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37314 (local address 127.9.123.124:39045)
0926 15:05:47.010244 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:47.010249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:47.010369 (+ 120us) server_negotiation.cc:407] Connection header received
0926 15:05:47.010581 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:47.010586 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:47.010651 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:47.010752 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:47.011698 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:47.012376 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:47.013610 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:47.014029 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:47.014153 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:47.017330 (+ 3177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:47.017335 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:47.017338 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:47.017370 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:47.020400 (+ 3030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:47.021079 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:47.021085 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:47.021088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:47.021287 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:47.021647 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:47.021652 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:47.021656 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:47.021974 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:47.022139 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:47.022577 (+ 438us) server_negotiation.cc:299] Negotiation successful
0926 15:05:47.022847 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 15:05:47.023746 23401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:47.010129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:39045 (local address 127.0.0.1:37314)
0926 15:05:47.010278 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:47.010299 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:47.010424 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:47.010800 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:47.010805 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:47.011385 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:05:47.011538 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:47.011548 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:47.012509 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:47.012515 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:47.013471 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:47.013484 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:47.014192 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:47.014198 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:47.014353 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:47.014855 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:05:47.014871 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:05:47.017146 (+ 2275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:05:47.020566 (+ 3420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:47.020570 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:47.020573 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:47.020948 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:47.021401 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:47.021406 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:47.021409 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:47.021540 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:47.022117 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:05:47.022122 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:05:47.022307 (+ 185us) client_negotiation.cc:769] Sending connection context
0926 15:05:47.022481 (+ 174us) client_negotiation.cc:240] Negotiation successful
0926 15:05:47.023341 (+ 860us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":222}
I20260926 15:05:47.024502 23326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:47.004479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34614 (local address 127.9.123.125:37155)
0926 15:05:47.005201 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:47.005206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:47.008322 (+ 3116us) server_negotiation.cc:407] Connection header received
0926 15:05:47.008665 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:47.008670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:47.008816 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:47.008952 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:47.010707 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:47.011471 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:47.012826 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:47.013348 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:47.013473 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:47.018036 (+ 4563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:47.018041 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:47.018044 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:47.018084 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:47.021128 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:47.021950 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:47.021956 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:47.021959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:47.022122 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:47.022671 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:47.022675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:47.022679 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:47.023073 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:47.023343 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:47.023766 (+ 423us) server_negotiation.cc:299] Negotiation successful
0926 15:05:47.024202 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":3636,"threads_started":1}
I20260926 15:05:47.029275 23396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb: Bootstrap starting.
I20260926 15:05:47.029805 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:05:47.032635 23396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb: Neither blocks nor log segments found. Creating new log.
I20260926 15:05:47.033838 23396 log.cc:824] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb: Log is configured to *not* fsync() on all Append() calls
I20260926 15:05:47.038182 23396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb: No bootstrap required, opened a new log
I20260926 15:05:47.049659 23396 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee7c64a76464fae99aafec5a4fe2c5f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39219 } } peers { permanent_uuid: "efa6b477c5f74291bad537cac10bb128" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } } peers { permanent_uuid: "5137e855a1954ebea5ad88ebe4a785cb" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } }
I20260926 15:05:47.050215 23396 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:05:47.050335 23396 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5137e855a1954ebea5ad88ebe4a785cb, State: Initialized, Role: FOLLOWER
I20260926 15:05:47.051107 23396 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee7c64a76464fae99aafec5a4fe2c5f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39219 } } peers { permanent_uuid: "efa6b477c5f74291bad537cac10bb128" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } } peers { permanent_uuid: "5137e855a1954ebea5ad88ebe4a785cb" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } }
I20260926 15:05:47.053632 23412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee7c64a76464fae99aafec5a4fe2c5f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39219 } } peers { permanent_uuid: "efa6b477c5f74291bad537cac10bb128" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } } peers { permanent_uuid: "5137e855a1954ebea5ad88ebe4a785cb" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } } }
I20260926 15:05:47.054072 23412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:05:47.055303 23396 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:05:47.055401 23402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:47.034834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55526 (local address 127.9.123.126:39219)
0926 15:05:47.035161 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:47.035168 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:47.036922 (+ 1754us) server_negotiation.cc:407] Connection header received
0926 15:05:47.037123 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:47.037128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:47.037219 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:47.037320 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:47.039844 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:47.040948 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:47.043361 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:47.044067 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:47.044174 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:47.048298 (+ 4124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:47.048307 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:47.048312 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:47.048365 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:47.051586 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:47.052717 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:47.052723 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:47.052726 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:47.052893 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:47.053458 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:47.053463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:47.053466 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:47.053861 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:47.054121 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:47.054663 (+ 542us) server_negotiation.cc:299] Negotiation successful
0926 15:05:47.055072 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260926 15:05:47.057056 23399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:47.035153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37318 (local address 127.9.123.124:39045)
0926 15:05:47.035517 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:47.035523 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:47.036138 (+ 615us) server_negotiation.cc:407] Connection header received
0926 15:05:47.036443 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:47.036448 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:47.036545 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:47.036625 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:47.039135 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:47.040253 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:47.042690 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:47.043399 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:47.043501 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:47.047803 (+ 4302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:47.047811 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:47.047819 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:47.047886 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:47.051676 (+ 3790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:47.052714 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:47.052719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:47.052722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:47.052882 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:47.055189 (+ 2307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:47.055196 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:47.055199 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:47.055613 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:47.055866 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:47.056424 (+ 558us) server_negotiation.cc:299] Negotiation successful
0926 15:05:47.056673 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179}
I20260926 15:05:47.064836 23248 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } attempt: 4
I20260926 15:05:47.071512 23326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:47.036444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34624 (local address 127.9.123.125:37155)
0926 15:05:47.036617 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:47.036622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:47.052337 (+ 15715us) server_negotiation.cc:407] Connection header received
0926 15:05:47.052660 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:47.052666 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:47.052762 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:47.052834 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:47.054965 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:47.056035 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:47.059062 (+ 3027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:47.059537 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:47.059684 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:47.063464 (+ 3780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:47.063472 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:47.063477 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:47.063521 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:47.066962 (+ 3441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:47.068040 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:47.068047 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:47.068052 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:47.068276 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:47.068788 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:47.068794 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:47.068798 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:47.069245 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:47.069493 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:47.070790 (+ 1297us) server_negotiation.cc:299] Negotiation successful
0926 15:05:47.071081 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Sep 26 15:05:47 dist-test-slave-7153 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435145, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for kudu/127.9.123.124@KRBTEST.COM
W20260926 15:05:47.077935 23423 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 15:05:47.078100 23423 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 15:05:47.065536 23329 master_runner.cc:186] Error getting master registration for 127.9.123.126:39219: OK, instance_id { permanent_uuid: "eee7c64a76464fae99aafec5a4fe2c5f" instance_seqno: 1790435145915615 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 15:05:47.078554 23329 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:05:47.084051 23399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:47.065987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37324 (local address 127.9.123.124:39045)
0926 15:05:47.066155 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:47.066161 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:47.066181 (+ 20us) server_negotiation.cc:407] Connection header received
0926 15:05:47.066234 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:47.066239 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:47.066314 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:47.066408 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:47.067439 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:47.068663 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:47.070511 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:47.071258 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:47.071399 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:47.076957 (+ 5558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:47.076965 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:47.076969 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:47.077039 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:47.080787 (+ 3748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:47.081874 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:47.081880 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:47.081884 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:47.082051 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:47.082453 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:47.082459 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:47.082462 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:47.082818 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:47.082966 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:47.083379 (+ 413us) server_negotiation.cc:299] Negotiation successful
0926 15:05:47.083620 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 15:05:47.084057 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:47.065342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:39045 (local address 127.0.0.1:37324)
0926 15:05:47.065668 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:47.065686 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:47.065784 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:47.066594 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:47.066599 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:47.067078 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:05:47.067273 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:47.067284 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:47.068836 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:47.068842 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:47.070332 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:47.070349 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:47.071627 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:47.071634 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:47.071833 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:47.072695 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:05:47.072721 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:05:47.076647 (+ 3926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:05:47.081074 (+ 4427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:47.081080 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:47.081084 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:47.081710 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:47.082173 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:47.082178 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:47.082183 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:47.082332 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:47.082942 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:05:47.082946 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:05:47.083123 (+ 177us) client_negotiation.cc:769] Sending connection context
0926 15:05:47.083256 (+ 133us) client_negotiation.cc:240] Negotiation successful
0926 15:05:47.083585 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204}
I20260926 15:05:47.089208 23248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eee7c64a76464fae99aafec5a4fe2c5f: Bootstrap starting.
I20260926 15:05:47.093511 23248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eee7c64a76464fae99aafec5a4fe2c5f: Neither blocks nor log segments found. Creating new log.
I20260926 15:05:47.094738 23248 log.cc:824] T 00000000000000000000000000000000 P eee7c64a76464fae99aafec5a4fe2c5f: Log is configured to *not* fsync() on all Append() calls
I20260926 15:05:47.098145 23248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eee7c64a76464fae99aafec5a4fe2c5f: No bootstrap required, opened a new log
I20260926 15:05:47.106776 23248 raft_consensus.cc:348] T 00000000000000000000000000000000 P eee7c64a76464fae99aafec5a4fe2c5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee7c64a76464fae99aafec5a4fe2c5f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39219 } } peers { permanent_uuid: "efa6b477c5f74291bad537cac10bb128" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } } peers { permanent_uuid: "5137e855a1954ebea5ad88ebe4a785cb" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } }
I20260926 15:05:47.107224 23248 raft_consensus.cc:374] T 00000000000000000000000000000000 P eee7c64a76464fae99aafec5a4fe2c5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:05:47.107354 23248 raft_consensus.cc:729] T 00000000000000000000000000000000 P eee7c64a76464fae99aafec5a4fe2c5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eee7c64a76464fae99aafec5a4fe2c5f, State: Initialized, Role: FOLLOWER
I20260926 15:05:47.108111 23248 consensus_queue.cc:260] T 00000000000000000000000000000000 P eee7c64a76464fae99aafec5a4fe2c5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee7c64a76464fae99aafec5a4fe2c5f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39219 } } peers { permanent_uuid: "efa6b477c5f74291bad537cac10bb128" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } } peers { permanent_uuid: "5137e855a1954ebea5ad88ebe4a785cb" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } }
I20260926 15:05:47.109196 23426 sys_catalog.cc:455] T 00000000000000000000000000000000 P eee7c64a76464fae99aafec5a4fe2c5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee7c64a76464fae99aafec5a4fe2c5f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39219 } } peers { permanent_uuid: "efa6b477c5f74291bad537cac10bb128" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } } peers { permanent_uuid: "5137e855a1954ebea5ad88ebe4a785cb" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } } }
I20260926 15:05:47.109675 23426 sys_catalog.cc:458] T 00000000000000000000000000000000 P eee7c64a76464fae99aafec5a4fe2c5f [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:05:47.110505 23248 sys_catalog.cc:564] T 00000000000000000000000000000000 P eee7c64a76464fae99aafec5a4fe2c5f [sys.catalog]: configured and running, proceeding with master startup.
W20260926 15:05:47.125458 23437 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eee7c64a76464fae99aafec5a4fe2c5f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 15:05:47.125696 23437 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 15:05:47.121223 23181 master_runner.cc:186] Error getting master registration for 127.9.123.125:37155: OK, instance_id { permanent_uuid: "efa6b477c5f74291bad537cac10bb128" instance_seqno: 1790435146422375 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 15:05:47.130481 23181 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:05:47.151842 23322 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } attempt: 5
W20260926 15:05:47.154102 23202 tablet.cc:2369] T 00000000000000000000000000000000 P eee7c64a76464fae99aafec5a4fe2c5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:05:47.161532 23412 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
Sep 26 15:05:47 dist-test-slave-7153 krb5kdc[23168](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435146, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.125@KRBTEST.COM for kudu/127.9.123.124@KRBTEST.COM
I20260926 15:05:47.161885 23412 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee7c64a76464fae99aafec5a4fe2c5f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39219 } } peers { permanent_uuid: "efa6b477c5f74291bad537cac10bb128" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } } peers { permanent_uuid: "5137e855a1954ebea5ad88ebe4a785cb" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } }
I20260926 15:05:47.163678 23412 leader_election.cc:288] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eee7c64a76464fae99aafec5a4fe2c5f (127.9.123.126:39219), efa6b477c5f74291bad537cac10bb128 (127.9.123.125:37155)
I20260926 15:05:47.165968 23223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5137e855a1954ebea5ad88ebe4a785cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eee7c64a76464fae99aafec5a4fe2c5f" is_pre_election: true
I20260926 15:05:47.166639 23223 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eee7c64a76464fae99aafec5a4fe2c5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5137e855a1954ebea5ad88ebe4a785cb in term 0.
I20260926 15:05:47.168164 23348 leader_election.cc:302] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5137e855a1954ebea5ad88ebe4a785cb, eee7c64a76464fae99aafec5a4fe2c5f; no voters:
I20260926 15:05:47.168694 23412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:05:47.168843 23412 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:05:47.168963 23412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:05:47.170804 23324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:47.152555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:39045 (local address 127.0.0.1:37326)
0926 15:05:47.152905 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:47.152934 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:47.153053 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:47.153462 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:47.153468 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:47.153998 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:05:47.154215 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:47.154227 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:47.155891 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:47.155898 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:47.157419 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:47.157438 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:47.158544 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:47.158551 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:47.158801 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:47.159502 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:05:47.159529 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:05:47.162697 (+ 3168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:05:47.167437 (+ 4740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:47.167442 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:47.167447 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:47.168024 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:47.168621 (+ 597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:47.168626 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:47.168630 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:47.168868 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:47.169756 (+ 888us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:05:47.169761 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:05:47.169946 (+ 185us) client_negotiation.cc:769] Sending connection context
0926 15:05:47.170086 (+ 140us) client_negotiation.cc:240] Negotiation successful
0926 15:05:47.170399 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217}
I20260926 15:05:47.172036 23399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:47.152759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37326 (local address 127.9.123.124:39045)
0926 15:05:47.152953 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:47.152959 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:47.152990 (+ 31us) server_negotiation.cc:407] Connection header received
0926 15:05:47.153226 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:47.153232 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:47.153314 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:47.153435 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:47.154495 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:47.155694 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:47.157597 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:47.158367 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:47.158514 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:47.162937 (+ 4423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:47.162944 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:47.162949 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:47.163005 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:47.167260 (+ 4255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:47.168182 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:47.168189 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:47.168193 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:47.168475 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:47.169094 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:47.169119 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:47.169122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:47.169471 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:47.169673 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:47.171227 (+ 1554us) server_negotiation.cc:299] Negotiation successful
0926 15:05:47.171612 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 15:05:47.172452 23412 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee7c64a76464fae99aafec5a4fe2c5f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39219 } } peers { permanent_uuid: "efa6b477c5f74291bad537cac10bb128" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } } peers { permanent_uuid: "5137e855a1954ebea5ad88ebe4a785cb" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } }
I20260926 15:05:47.174721 23223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5137e855a1954ebea5ad88ebe4a785cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eee7c64a76464fae99aafec5a4fe2c5f"
I20260926 15:05:47.174985 23223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eee7c64a76464fae99aafec5a4fe2c5f [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:05:47.178365 23223 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eee7c64a76464fae99aafec5a4fe2c5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5137e855a1954ebea5ad88ebe4a785cb in term 1.
W20260926 15:05:47.179100 23350 tablet.cc:2369] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:05:47.180068 23322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efa6b477c5f74291bad537cac10bb128: Bootstrap starting.
I20260926 15:05:47.180474 23348 leader_election.cc:302] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5137e855a1954ebea5ad88ebe4a785cb, eee7c64a76464fae99aafec5a4fe2c5f; no voters:
I20260926 15:05:47.182747 23412 leader_election.cc:288] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [CANDIDATE]: Term 1 election: Requested vote from peers eee7c64a76464fae99aafec5a4fe2c5f (127.9.123.126:39219), efa6b477c5f74291bad537cac10bb128 (127.9.123.125:37155)
I20260926 15:05:47.183661 23412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:05:47.184309 23412 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [term 1 LEADER]: Becoming Leader. State: Replica: 5137e855a1954ebea5ad88ebe4a785cb, State: Running, Role: LEADER
I20260926 15:05:47.185227 23322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efa6b477c5f74291bad537cac10bb128: Neither blocks nor log segments found. Creating new log.
I20260926 15:05:47.186793 23322 log.cc:824] T 00000000000000000000000000000000 P efa6b477c5f74291bad537cac10bb128: Log is configured to *not* fsync() on all Append() calls
I20260926 15:05:47.187139 23412 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee7c64a76464fae99aafec5a4fe2c5f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39219 } } peers { permanent_uuid: "efa6b477c5f74291bad537cac10bb128" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } } peers { permanent_uuid: "5137e855a1954ebea5ad88ebe4a785cb" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } }
I20260926 15:05:47.197006 23322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efa6b477c5f74291bad537cac10bb128: No bootstrap required, opened a new log
I20260926 15:05:47.200873 23439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5137e855a1954ebea5ad88ebe4a785cb. Latest consensus state: current_term: 1 leader_uuid: "5137e855a1954ebea5ad88ebe4a785cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee7c64a76464fae99aafec5a4fe2c5f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39219 } } peers { permanent_uuid: "efa6b477c5f74291bad537cac10bb128" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } } peers { permanent_uuid: "5137e855a1954ebea5ad88ebe4a785cb" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } } }
I20260926 15:05:47.201442 23439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [sys.catalog]: This master's current role is: LEADER
I20260926 15:05:47.207999 23442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:05:47.212390 23322 raft_consensus.cc:348] T 00000000000000000000000000000000 P efa6b477c5f74291bad537cac10bb128 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee7c64a76464fae99aafec5a4fe2c5f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39219 } } peers { permanent_uuid: "efa6b477c5f74291bad537cac10bb128" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } } peers { permanent_uuid: "5137e855a1954ebea5ad88ebe4a785cb" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } }
I20260926 15:05:47.212818 23322 raft_consensus.cc:374] T 00000000000000000000000000000000 P efa6b477c5f74291bad537cac10bb128 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:05:47.212940 23322 raft_consensus.cc:729] T 00000000000000000000000000000000 P efa6b477c5f74291bad537cac10bb128 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa6b477c5f74291bad537cac10bb128, State: Initialized, Role: FOLLOWER
I20260926 15:05:47.213624 23442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:05:47.213832 23322 consensus_queue.cc:260] T 00000000000000000000000000000000 P efa6b477c5f74291bad537cac10bb128 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee7c64a76464fae99aafec5a4fe2c5f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39219 } } peers { permanent_uuid: "efa6b477c5f74291bad537cac10bb128" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } } peers { permanent_uuid: "5137e855a1954ebea5ad88ebe4a785cb" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } }
I20260926 15:05:47.229774 23443 sys_catalog.cc:455] T 00000000000000000000000000000000 P efa6b477c5f74291bad537cac10bb128 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee7c64a76464fae99aafec5a4fe2c5f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39219 } } peers { permanent_uuid: "efa6b477c5f74291bad537cac10bb128" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } } peers { permanent_uuid: "5137e855a1954ebea5ad88ebe4a785cb" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } } }
I20260926 15:05:47.230436 23443 sys_catalog.cc:458] T 00000000000000000000000000000000 P efa6b477c5f74291bad537cac10bb128 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:05:47.231473 23322 sys_catalog.cc:564] T 00000000000000000000000000000000 P efa6b477c5f74291bad537cac10bb128 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:05:47.231555 23297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5137e855a1954ebea5ad88ebe4a785cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa6b477c5f74291bad537cac10bb128" is_pre_election: true
I20260926 15:05:47.232277 23297 raft_consensus.cc:2454] T 00000000000000000000000000000000 P efa6b477c5f74291bad537cac10bb128 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5137e855a1954ebea5ad88ebe4a785cb in term 0.
I20260926 15:05:47.231501 23296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5137e855a1954ebea5ad88ebe4a785cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa6b477c5f74291bad537cac10bb128"
I20260926 15:05:47.233639 23296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efa6b477c5f74291bad537cac10bb128 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:05:47.237517 23296 raft_consensus.cc:2454] T 00000000000000000000000000000000 P efa6b477c5f74291bad537cac10bb128 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5137e855a1954ebea5ad88ebe4a785cb in term 1.
I20260926 15:05:47.246017 23296 raft_consensus.cc:1261] T 00000000000000000000000000000000 P efa6b477c5f74291bad537cac10bb128 [term 1 FOLLOWER]: Refusing update from remote peer 5137e855a1954ebea5ad88ebe4a785cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:05:47.247155 23439 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa6b477c5f74291bad537cac10bb128" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:05:47.254449 23223 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eee7c64a76464fae99aafec5a4fe2c5f [term 1 FOLLOWER]: Refusing update from remote peer 5137e855a1954ebea5ad88ebe4a785cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:05:47.255543 23439 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "eee7c64a76464fae99aafec5a4fe2c5f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:05:47.283459 23426 sys_catalog.cc:455] T 00000000000000000000000000000000 P eee7c64a76464fae99aafec5a4fe2c5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5137e855a1954ebea5ad88ebe4a785cb. Latest consensus state: current_term: 1 leader_uuid: "5137e855a1954ebea5ad88ebe4a785cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee7c64a76464fae99aafec5a4fe2c5f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39219 } } peers { permanent_uuid: "efa6b477c5f74291bad537cac10bb128" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } } peers { permanent_uuid: "5137e855a1954ebea5ad88ebe4a785cb" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } } }
I20260926 15:05:47.285419 23426 sys_catalog.cc:458] T 00000000000000000000000000000000 P eee7c64a76464fae99aafec5a4fe2c5f [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:05:47.287506 23443 sys_catalog.cc:455] T 00000000000000000000000000000000 P efa6b477c5f74291bad537cac10bb128 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5137e855a1954ebea5ad88ebe4a785cb. Latest consensus state: current_term: 1 leader_uuid: "5137e855a1954ebea5ad88ebe4a785cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee7c64a76464fae99aafec5a4fe2c5f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39219 } } peers { permanent_uuid: "efa6b477c5f74291bad537cac10bb128" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } } peers { permanent_uuid: "5137e855a1954ebea5ad88ebe4a785cb" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } } }
I20260926 15:05:47.287895 23443 sys_catalog.cc:458] T 00000000000000000000000000000000 P efa6b477c5f74291bad537cac10bb128 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:05:47.288394 23444 mvcc.cc:205] Tried to move back new op lower bound from 7333622363105710080 to 7333622362897096704. Current Snapshot: MvccSnapshot[applied={T|T < 7333622363105710080}]
I20260926 15:05:47.290364 23446 mvcc.cc:205] Tried to move back new op lower bound from 7333622363105710080 to 7333622362897096704. Current Snapshot: MvccSnapshot[applied={T|T < 7333622363105710080}]
I20260926 15:05:47.290897 23442 catalog_manager.cc:1347] Generated new cluster ID: 707020cb56714bec88bf1b594fa05e14
I20260926 15:05:47.291007 23442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:05:47.291033 23439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5137e855a1954ebea5ad88ebe4a785cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee7c64a76464fae99aafec5a4fe2c5f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39219 } } peers { permanent_uuid: "efa6b477c5f74291bad537cac10bb128" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } } peers { permanent_uuid: "5137e855a1954ebea5ad88ebe4a785cb" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } } }
I20260926 15:05:47.291282 23439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [sys.catalog]: This master's current role is: LEADER
I20260926 15:05:47.291837 23443 sys_catalog.cc:455] T 00000000000000000000000000000000 P efa6b477c5f74291bad537cac10bb128 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5137e855a1954ebea5ad88ebe4a785cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee7c64a76464fae99aafec5a4fe2c5f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39219 } } peers { permanent_uuid: "efa6b477c5f74291bad537cac10bb128" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } } peers { permanent_uuid: "5137e855a1954ebea5ad88ebe4a785cb" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } } }
I20260926 15:05:47.292083 23443 sys_catalog.cc:458] T 00000000000000000000000000000000 P efa6b477c5f74291bad537cac10bb128 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:05:47.311066 23412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5137e855a1954ebea5ad88ebe4a785cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee7c64a76464fae99aafec5a4fe2c5f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39219 } } peers { permanent_uuid: "efa6b477c5f74291bad537cac10bb128" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } } peers { permanent_uuid: "5137e855a1954ebea5ad88ebe4a785cb" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } } }
I20260926 15:05:47.311441 23412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb [sys.catalog]: This master's current role is: LEADER
I20260926 15:05:47.313251 23447 mvcc.cc:205] Tried to move back new op lower bound from 7333622363105710080 to 7333622362897096704. Current Snapshot: MvccSnapshot[applied={T|T < 7333622363105710080}]
I20260926 15:05:47.322845 23442 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:05:47.324756 23426 sys_catalog.cc:455] T 00000000000000000000000000000000 P eee7c64a76464fae99aafec5a4fe2c5f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5137e855a1954ebea5ad88ebe4a785cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee7c64a76464fae99aafec5a4fe2c5f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39219 } } peers { permanent_uuid: "efa6b477c5f74291bad537cac10bb128" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 37155 } } peers { permanent_uuid: "5137e855a1954ebea5ad88ebe4a785cb" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 39045 } } }
I20260926 15:05:47.325057 23426 sys_catalog.cc:458] T 00000000000000000000000000000000 P eee7c64a76464fae99aafec5a4fe2c5f [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:05:47.331646 23442 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:05:47.353453 23462 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 15:05:47.353966 23442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb: Generated new TSK 0
I20260926 15:05:47.355177 23442 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:05:48.127811 23437 catalog_manager.cc:1259] Loaded cluster ID: 707020cb56714bec88bf1b594fa05e14
I20260926 15:05:48.127920 23437 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eee7c64a76464fae99aafec5a4fe2c5f: loading cluster ID for follower catalog manager: success
I20260926 15:05:48.130520 23437 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eee7c64a76464fae99aafec5a4fe2c5f: acquiring CA information for follower catalog manager: success
I20260926 15:05:48.132916 23437 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eee7c64a76464fae99aafec5a4fe2c5f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 15:05:48.357918 23462 catalog_manager.cc:1259] Loaded cluster ID: 707020cb56714bec88bf1b594fa05e14
I20260926 15:05:48.358029 23462 catalog_manager.cc:1552] T 00000000000000000000000000000000 P efa6b477c5f74291bad537cac10bb128: loading cluster ID for follower catalog manager: success
I20260926 15:05:48.360818 23462 catalog_manager.cc:1574] T 00000000000000000000000000000000 P efa6b477c5f74291bad537cac10bb128: acquiring CA information for follower catalog manager: success
I20260926 15:05:48.362893 23462 catalog_manager.cc:1602] T 00000000000000000000000000000000 P efa6b477c5f74291bad537cac10bb128: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 15:05:48.460160 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 23181
I20260926 15:05:48.461476 23243 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:05:48.573771 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 23181
I20260926 15:05:48.591795 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 23255
I20260926 15:05:48.593040 23317 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:05:48.646210 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:48.645415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39219 (local address 127.0.0.1:55534)
0926 15:05:48.645902 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:48.646041 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":262,"threads_started":1}
W20260926 15:05:48.646497 23348 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39219: connect: Connection refused (error 111)
I20260926 15:05:48.650285 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:48.649906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39219 (local address 127.0.0.1:55540)
0926 15:05:48.650068 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:48.650147 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260926 15:05:48.650487 23348 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5137e855a1954ebea5ad88ebe4a785cb -> Peer eee7c64a76464fae99aafec5a4fe2c5f (127.9.123.126:39219): Couldn't send request to peer eee7c64a76464fae99aafec5a4fe2c5f. Status: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39219: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260926 15:05:48.706996 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 23255
I20260926 15:05:48.722781 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 23329
I20260926 15:05:48.723979 23391 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:05:48.837005 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 23329
2026-09-26T15:05:48Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4647 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:05:48 dist-test-slave-7153 krb5kdc[23474](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:05:48 dist-test-slave-7153 krb5kdc[23474](info): set up 2 sockets
Sep 26 15:05:48 dist-test-slave-7153 krb5kdc[23474](info): commencing operation
krb5kdc: starting...
W20260926 15:05:49.893752 9709 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:05:50 dist-test-slave-7153 krb5kdc[23474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:05:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:05:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:05:50.130649 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:36605
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:40101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:36605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:05:50.482586 23487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:05:50.482913 23487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:05:50.483035 23487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:05:50.490669 23487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:05:50.490811 23487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:05:50.490863 23487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:05:50.490940 23487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:05:50.490998 23487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:05:50.501456 23487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:40101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:36605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:36605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:05:50.502875 23487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:05:50.504307 23487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:05:50.516508 23492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:50.516516 23493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:50.516558 23495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:50.517870 23487 server_base.cc:1034] running on GCE node
I20260926 15:05:50.518712 23487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:05:50.520303 23487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:05:50.521519 23487 hybrid_clock.cc:648] HybridClock initialized: now 1790435150521470 us; error 34 us; skew 500 ppm
I20260926 15:05:50.521899 23487 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:05:50 dist-test-slave-7153 krb5kdc[23474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435150, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:50.525200 23487 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:05:50.526921 23487 webserver.cc:460] Webserver started at http://127.9.123.126:37241/ using document root <none> and password file <none>
I20260926 15:05:50.527678 23487 fs_manager.cc:359] Metadata directory not provided
I20260926 15:05:50.527806 23487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:05:50.528126 23487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:05:50.531561 23487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "0d1ce67b714a454192b69123933f667f"
format_stamp: "Formatted at 2026-09-26 15:05:50 on dist-test-slave-7153"
server_key: "ff4a997bab0a67fcbf9d81e027bd5a42"
server_key_iv: "934ed59e20a1356a42b94596acbee71f"
server_key_version: "encryptionkey@0"
I20260926 15:05:50.532419 23487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "0d1ce67b714a454192b69123933f667f"
format_stamp: "Formatted at 2026-09-26 15:05:50 on dist-test-slave-7153"
server_key: "ff4a997bab0a67fcbf9d81e027bd5a42"
server_key_iv: "934ed59e20a1356a42b94596acbee71f"
server_key_version: "encryptionkey@0"
I20260926 15:05:50.538228 23487 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 15:05:50.542272 23502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:50.543828 23487 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 15:05:50.544003 23487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "0d1ce67b714a454192b69123933f667f"
format_stamp: "Formatted at 2026-09-26 15:05:50 on dist-test-slave-7153"
server_key: "ff4a997bab0a67fcbf9d81e027bd5a42"
server_key_iv: "934ed59e20a1356a42b94596acbee71f"
server_key_version: "encryptionkey@0"
I20260926 15:05:50.544216 23487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:05:50.597013 23487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:05:50.601810 23487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:05:50.602115 23487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:05:50.619652 23487 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:36605
I20260926 15:05:50.619673 23553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:36605 every 8 connection(s)
I20260926 15:05:50.621091 23487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:05:50.624647 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 23487
I20260926 15:05:50.624846 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:05:50.625232 23554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:05:50.625554 9709 external_mini_cluster.cc:1428] Setting key d560b35181204dd695b7abca0d977068
I20260926 15:05:50.640628 23554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d1ce67b714a454192b69123933f667f: Bootstrap starting.
Sep 26 15:05:50 dist-test-slave-7153 krb5kdc[23474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:05:50.644532 23554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d1ce67b714a454192b69123933f667f: Neither blocks nor log segments found. Creating new log.
I20260926 15:05:50.645742 23554 log.cc:824] T 00000000000000000000000000000000 P 0d1ce67b714a454192b69123933f667f: Log is configured to *not* fsync() on all Append() calls
I20260926 15:05:50.649288 23554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d1ce67b714a454192b69123933f667f: No bootstrap required, opened a new log
I20260926 15:05:50.650242 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:50.627486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53778 (local address 127.9.123.126:36605)
0926 15:05:50.628341 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:50.628363 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:50.628417 (+ 54us) server_negotiation.cc:407] Connection header received
0926 15:05:50.630427 (+ 2010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:50.630462 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:50.631063 (+ 601us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:50.631578 (+ 515us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:50.633085 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:50.635184 (+ 2099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:50.636624 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:50.637781 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:50.637998 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:50.642314 (+ 4316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:50.642327 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:50.642344 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:50.642397 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:50.645556 (+ 3159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:50.646625 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:50.646635 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:50.646643 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:50.646875 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:50.647327 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:50.647331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:50.647334 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:50.647922 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:50.648171 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:50.648691 (+ 520us) server_negotiation.cc:299] Negotiation successful
0926 15:05:50.649091 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":331,"threads_started":1}
I20260926 15:05:50.654718 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:05:50.657953 23554 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d1ce67b714a454192b69123933f667f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1ce67b714a454192b69123933f667f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 36605 } }
I20260926 15:05:50.658349 23554 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d1ce67b714a454192b69123933f667f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:05:50.658460 23554 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d1ce67b714a454192b69123933f667f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d1ce67b714a454192b69123933f667f, State: Initialized, Role: FOLLOWER
I20260926 15:05:50.659058 23554 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d1ce67b714a454192b69123933f667f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1ce67b714a454192b69123933f667f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 36605 } }
I20260926 15:05:50.659291 23554 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d1ce67b714a454192b69123933f667f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:05:50.659420 23554 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d1ce67b714a454192b69123933f667f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:05:50.659569 23554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d1ce67b714a454192b69123933f667f [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:05:50.662017 23554 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d1ce67b714a454192b69123933f667f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1ce67b714a454192b69123933f667f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 36605 } }
I20260926 15:05:50.662568 23554 leader_election.cc:302] T 00000000000000000000000000000000 P 0d1ce67b714a454192b69123933f667f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d1ce67b714a454192b69123933f667f; no voters:
I20260926 15:05:50.663225 23554 leader_election.cc:288] T 00000000000000000000000000000000 P 0d1ce67b714a454192b69123933f667f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:05:50.663384 23559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d1ce67b714a454192b69123933f667f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:05:50.664018 23559 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d1ce67b714a454192b69123933f667f [term 1 LEADER]: Becoming Leader. State: Replica: 0d1ce67b714a454192b69123933f667f, State: Running, Role: LEADER
I20260926 15:05:50.664611 23559 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d1ce67b714a454192b69123933f667f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1ce67b714a454192b69123933f667f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 36605 } }
I20260926 15:05:50.665282 23554 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d1ce67b714a454192b69123933f667f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:05:50.667737 23561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d1ce67b714a454192b69123933f667f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d1ce67b714a454192b69123933f667f. Latest consensus state: current_term: 1 leader_uuid: "0d1ce67b714a454192b69123933f667f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1ce67b714a454192b69123933f667f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 36605 } } }
I20260926 15:05:50.668172 23561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d1ce67b714a454192b69123933f667f [sys.catalog]: This master's current role is: LEADER
I20260926 15:05:50.667738 23560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d1ce67b714a454192b69123933f667f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d1ce67b714a454192b69123933f667f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d1ce67b714a454192b69123933f667f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 36605 } } }
I20260926 15:05:50.668728 23560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d1ce67b714a454192b69123933f667f [sys.catalog]: This master's current role is: LEADER
I20260926 15:05:50.669809 23568 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:05:50.674831 23568 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:05:50.684151 23568 catalog_manager.cc:1347] Generated new cluster ID: 6c31ed676da74ff0b47ce109bfb1ee6e
I20260926 15:05:50.684296 23568 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:05:50.707540 23568 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:05:50.709288 23568 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:05:50.717552 23568 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d1ce67b714a454192b69123933f667f: Generated new TSK 0
I20260926 15:05:50.718518 23568 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:05:50.767504 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 23487
I20260926 15:05:50.768836 23549 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:05:50.868013 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 23487
2026-09-26T15:05:50Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2027 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:05:50 dist-test-slave-7153 krb5kdc[23582](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:05:50 dist-test-slave-7153 krb5kdc[23582](info): set up 2 sockets
Sep 26 15:05:50 dist-test-slave-7153 krb5kdc[23582](info): commencing operation
krb5kdc: starting...
W20260926 15:05:51.893646 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.991s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:05:52 dist-test-slave-7153 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:05:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:05:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:05:52.126222 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:39699
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:44657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:39699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:05:52.466754 23595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:05:52.467168 23595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:05:52.467273 23595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:05:52.475201 23595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:05:52.475350 23595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:05:52.475396 23595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:05:52.475453 23595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:05:52.475495 23595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:05:52.485880 23595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:44657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:39699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:39699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:05:52.487186 23595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:05:52.488557 23595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:05:52.500871 23603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:52.500923 23601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:52.500923 23600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:52.501742 23595 server_base.cc:1034] running on GCE node
I20260926 15:05:52.502746 23595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:05:52.504369 23595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:05:52.505636 23595 hybrid_clock.cc:648] HybridClock initialized: now 1790435152505569 us; error 63 us; skew 500 ppm
I20260926 15:05:52.506031 23595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:05:52 dist-test-slave-7153 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435152, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:52.509598 23595 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:05:52.511332 23595 webserver.cc:460] Webserver started at http://127.9.123.126:34565/ using document root <none> and password file <none>
I20260926 15:05:52.512058 23595 fs_manager.cc:359] Metadata directory not provided
I20260926 15:05:52.512182 23595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:05:52.512480 23595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:05:52.516110 23595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "f1b5473193be4bba9ed73a4ef5eae05b"
format_stamp: "Formatted at 2026-09-26 15:05:52 on dist-test-slave-7153"
server_key: "ef60863b8c1e4db4b0c7973ace27d81e"
server_key_iv: "23aaed99722434801033d129cbbaee7d"
server_key_version: "encryptionkey@0"
I20260926 15:05:52.516942 23595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "f1b5473193be4bba9ed73a4ef5eae05b"
format_stamp: "Formatted at 2026-09-26 15:05:52 on dist-test-slave-7153"
server_key: "ef60863b8c1e4db4b0c7973ace27d81e"
server_key_iv: "23aaed99722434801033d129cbbaee7d"
server_key_version: "encryptionkey@0"
I20260926 15:05:52.523382 23595 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 15:05:52.527431 23610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:52.529222 23595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 15:05:52.529425 23595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "f1b5473193be4bba9ed73a4ef5eae05b"
format_stamp: "Formatted at 2026-09-26 15:05:52 on dist-test-slave-7153"
server_key: "ef60863b8c1e4db4b0c7973ace27d81e"
server_key_iv: "23aaed99722434801033d129cbbaee7d"
server_key_version: "encryptionkey@0"
I20260926 15:05:52.529639 23595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:05:52.550912 23595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:05:52.555250 23595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:05:52.555545 23595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:05:52.573920 23595 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:39699
I20260926 15:05:52.573954 23661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:39699 every 8 connection(s)
I20260926 15:05:52.575340 23595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:05:52.579277 23662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:05:52.579427 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 23595
I20260926 15:05:52.579623 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:05:52.580295 9709 external_mini_cluster.cc:1428] Setting key c54aac11a634679e9aedbd10e40df234
I20260926 15:05:52.594277 23662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b: Bootstrap starting.
Sep 26 15:05:52 dist-test-slave-7153 krb5kdc[23582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:05:52.597857 23662 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b: Neither blocks nor log segments found. Creating new log.
I20260926 15:05:52.599318 23662 log.cc:824] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b: Log is configured to *not* fsync() on all Append() calls
I20260926 15:05:52.603207 23662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b: No bootstrap required, opened a new log
I20260926 15:05:52.607977 23665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:52.582379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57898 (local address 127.9.123.126:39699)
0926 15:05:52.583167 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:52.583187 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:52.583239 (+ 52us) server_negotiation.cc:407] Connection header received
0926 15:05:52.585960 (+ 2721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:52.585997 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:52.586618 (+ 621us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:52.587129 (+ 511us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:52.588611 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:52.590833 (+ 2222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:52.592318 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:52.593447 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:52.593660 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:52.598217 (+ 4557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:52.598231 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:52.598260 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:52.598331 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:52.602346 (+ 4015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:52.603295 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:52.603304 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:52.603314 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:52.603550 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:52.603963 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:52.603968 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:52.603971 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:52.604770 (+ 799us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:52.605140 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:52.605854 (+ 714us) server_negotiation.cc:299] Negotiation successful
0926 15:05:52.606423 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":299,"threads_started":1}
I20260926 15:05:52.613301 23662 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b5473193be4bba9ed73a4ef5eae05b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39699 } }
I20260926 15:05:52.613715 23662 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:05:52.613794 23662 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1b5473193be4bba9ed73a4ef5eae05b, State: Initialized, Role: FOLLOWER
I20260926 15:05:52.614349 23662 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b5473193be4bba9ed73a4ef5eae05b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39699 } }
I20260926 15:05:52.614570 23662 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:05:52.614686 23662 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:05:52.614823 23662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:05:52.617167 23662 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b5473193be4bba9ed73a4ef5eae05b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39699 } }
I20260926 15:05:52.617786 23662 leader_election.cc:302] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1b5473193be4bba9ed73a4ef5eae05b; no voters:
I20260926 15:05:52.618515 23662 leader_election.cc:288] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:05:52.618644 23667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:05:52.619282 23667 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 1 LEADER]: Becoming Leader. State: Replica: f1b5473193be4bba9ed73a4ef5eae05b, State: Running, Role: LEADER
I20260926 15:05:52.619992 23667 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b5473193be4bba9ed73a4ef5eae05b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39699 } }
I20260926 15:05:52.620947 23662 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:05:52.623032 23669 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1b5473193be4bba9ed73a4ef5eae05b. Latest consensus state: current_term: 1 leader_uuid: "f1b5473193be4bba9ed73a4ef5eae05b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b5473193be4bba9ed73a4ef5eae05b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39699 } } }
I20260926 15:05:52.623497 23669 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [sys.catalog]: This master's current role is: LEADER
I20260926 15:05:52.625510 23668 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1b5473193be4bba9ed73a4ef5eae05b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b5473193be4bba9ed73a4ef5eae05b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39699 } } }
I20260926 15:05:52.625979 23668 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [sys.catalog]: This master's current role is: LEADER
I20260926 15:05:52.627713 23673 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:05:52.633332 23673 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:05:52.642493 23673 catalog_manager.cc:1347] Generated new cluster ID: 07a465c2cd634c1e8cc51a2cb3135d52
I20260926 15:05:52.642609 23673 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:05:52.676064 23673 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:05:52.677579 23673 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:05:52.687572 23673 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b: Generated new TSK 0
I20260926 15:05:52.688732 23673 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.123.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
I20260926 15:05:52.755151 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:39699
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:44657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:05:53.101547 23688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:05:53.101856 23688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:05:53.101958 23688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:05:53.109870 23688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:05:53.110023 23688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:05:53.110152 23688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:05:53.120626 23688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:44657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:39699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:05:53.122170 23688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:05:53.123574 23688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:05:53.137656 23694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:53.137683 23693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:53.137634 23696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:53.138603 23688 server_base.cc:1034] running on GCE node
I20260926 15:05:53.139214 23688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:05:53.140226 23688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:05:53.141445 23688 hybrid_clock.cc:648] HybridClock initialized: now 1790435153141382 us; error 56 us; skew 500 ppm
I20260926 15:05:53.141777 23688 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:05:53 dist-test-slave-7153 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435153, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:53.145182 23688 init.cc:377] Logged in from keytab as kudu/127.9.123.65@KRBTEST.COM (short username kudu)
I20260926 15:05:53.146687 23688 webserver.cc:460] Webserver started at http://127.9.123.65:37973/ using document root <none> and password file <none>
I20260926 15:05:53.147405 23688 fs_manager.cc:359] Metadata directory not provided
I20260926 15:05:53.147523 23688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:05:53.147821 23688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:05:53.151541 23688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "b4a122d72de34473b78cd74c6086d1dc"
format_stamp: "Formatted at 2026-09-26 15:05:53 on dist-test-slave-7153"
server_key: "44fa0d2ab79b8b5a5f35ff3e7ba3e1fc"
server_key_iv: "03c41bc14fe4f82f0827b1b60b7cfd43"
server_key_version: "encryptionkey@0"
I20260926 15:05:53.152456 23688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "b4a122d72de34473b78cd74c6086d1dc"
format_stamp: "Formatted at 2026-09-26 15:05:53 on dist-test-slave-7153"
server_key: "44fa0d2ab79b8b5a5f35ff3e7ba3e1fc"
server_key_iv: "03c41bc14fe4f82f0827b1b60b7cfd43"
server_key_version: "encryptionkey@0"
I20260926 15:05:53.158538 23688 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260926 15:05:53.162473 23703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:53.164234 23688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 15:05:53.164443 23688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "b4a122d72de34473b78cd74c6086d1dc"
format_stamp: "Formatted at 2026-09-26 15:05:53 on dist-test-slave-7153"
server_key: "44fa0d2ab79b8b5a5f35ff3e7ba3e1fc"
server_key_iv: "03c41bc14fe4f82f0827b1b60b7cfd43"
server_key_version: "encryptionkey@0"
I20260926 15:05:53.164669 23688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:05:53.189659 23688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:05:53.194352 23688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:05:53.194659 23688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:05:53.195679 23688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:05:53.197176 23688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:05:53.197261 23688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:53.197386 23688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:05:53.197439 23688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:53.230185 23688 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:37823
I20260926 15:05:53.230238 23815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:37823 every 8 connection(s)
I20260926 15:05:53.231570 23688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:05:53.239202 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 23688
I20260926 15:05:53.239413 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:05:53.240098 9709 external_mini_cluster.cc:1428] Setting key 6ed027009db1a170751fd5145189cbd6
Sep 26 15:05:53 dist-test-slave-7153 krb5kdc[23582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435153, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:05:53.253407 23818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:53.234346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:58239 (local address 127.9.123.126:39699)
0926 15:05:53.234822 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:53.234827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:53.235832 (+ 1005us) server_negotiation.cc:407] Connection header received
0926 15:05:53.237873 (+ 2041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:53.237880 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:53.237978 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:53.238077 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:53.239894 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:53.240700 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:53.242648 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:53.243070 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:53.243200 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:53.246922 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:53.246928 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:53.246932 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:53.246970 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:53.249641 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:53.250578 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:53.250585 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:53.250589 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:53.250761 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:53.251383 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:53.251390 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:53.251393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:53.251713 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:53.251868 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:53.252732 (+ 864us) server_negotiation.cc:299] Negotiation successful
0926 15:05:53.252998 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":245,"threads_started":1}
I20260926 15:05:53.254513 23819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:53.234869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.65:58239)
0926 15:05:53.235631 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:53.235681 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:53.237548 (+ 1867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:53.238371 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:53.238381 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:53.238954 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:05:53.239694 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:53.239717 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:53.240877 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:53.240882 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:53.242480 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:53.242497 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:53.243326 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:53.243334 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:53.243578 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:53.244271 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:05:53.244296 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:05:53.246647 (+ 2351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:05:53.249831 (+ 3184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:53.249843 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:53.249858 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:53.250391 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:53.250890 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:53.250894 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:53.250896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:53.251203 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:53.251871 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:05:53.251881 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:05:53.252311 (+ 430us) client_negotiation.cc:769] Sending connection context
0926 15:05:53.252578 (+ 267us) client_negotiation.cc:240] Negotiation successful
0926 15:05:53.253227 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":295,"threads_started":1}
I20260926 15:05:53.256419 23816 heartbeater.cc:347] Connected to a master server at 127.9.123.126:39699
I20260926 15:05:53.256758 23816 heartbeater.cc:464] Registering TS with master...
I20260926 15:05:53.257566 23816 heartbeater.cc:511] Master 127.9.123.126:39699 requested a full tablet report, sending...
I20260926 15:05:53.260174 23627 ts_manager.cc:194] Registered new tserver with Master: b4a122d72de34473b78cd74c6086d1dc (127.9.123.65:37823)
WARNING: no policy specified for kudu/127.9.123.66@KRBTEST.COM; defaulting to no policy
I20260926 15:05:53.262224 23627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.65@KRBTEST.COM'} at 127.9.123.65:58239
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
I20260926 15:05:53.292343 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:39699
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:44657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:05:53.629073 23822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:05:53.629434 23822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:05:53.629540 23822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:05:53.637593 23822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:05:53.637738 23822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:05:53.637873 23822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:05:53.648299 23822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:44657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:39699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:05:53.649816 23822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:05:53.651206 23822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:05:53.665138 23828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:53.665316 23830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:53.665496 23827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:53.665935 23822 server_base.cc:1034] running on GCE node
I20260926 15:05:53.666558 23822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:05:53.667541 23822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:05:53.668766 23822 hybrid_clock.cc:648] HybridClock initialized: now 1790435153668704 us; error 51 us; skew 500 ppm
I20260926 15:05:53.669227 23822 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:05:53 dist-test-slave-7153 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435153, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:53.672483 23822 init.cc:377] Logged in from keytab as kudu/127.9.123.66@KRBTEST.COM (short username kudu)
I20260926 15:05:53.674132 23822 webserver.cc:460] Webserver started at http://127.9.123.66:42239/ using document root <none> and password file <none>
I20260926 15:05:53.674871 23822 fs_manager.cc:359] Metadata directory not provided
I20260926 15:05:53.674984 23822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:05:53.675307 23822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:05:53.678459 23822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "cd61584d310b455ba36e80f6178ec37b"
format_stamp: "Formatted at 2026-09-26 15:05:53 on dist-test-slave-7153"
server_key: "8c56bbf1e9883c429769438d8aedb809"
server_key_iv: "3446ffbc19b1b57693716e0a95e74c82"
server_key_version: "encryptionkey@0"
I20260926 15:05:53.679284 23822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "cd61584d310b455ba36e80f6178ec37b"
format_stamp: "Formatted at 2026-09-26 15:05:53 on dist-test-slave-7153"
server_key: "8c56bbf1e9883c429769438d8aedb809"
server_key_iv: "3446ffbc19b1b57693716e0a95e74c82"
server_key_version: "encryptionkey@0"
I20260926 15:05:53.685046 23822 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 15:05:53.689078 23837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:53.690847 23822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:05:53.691054 23822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "cd61584d310b455ba36e80f6178ec37b"
format_stamp: "Formatted at 2026-09-26 15:05:53 on dist-test-slave-7153"
server_key: "8c56bbf1e9883c429769438d8aedb809"
server_key_iv: "3446ffbc19b1b57693716e0a95e74c82"
server_key_version: "encryptionkey@0"
I20260926 15:05:53.691284 23822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:05:53.705446 23822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:05:53.710870 23822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:05:53.711257 23822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:05:53.712387 23822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:05:53.714314 23822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:05:53.714408 23822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:53.714524 23822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:05:53.714593 23822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:53.751957 23822 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:45037
I20260926 15:05:53.752013 23949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:45037 every 8 connection(s)
I20260926 15:05:53.753444 23822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:05:53.755602 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 23822
I20260926 15:05:53.756009 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:05:53.756678 9709 external_mini_cluster.cc:1428] Setting key a67c91dbc3a21668bd4369a7a0c79223
Sep 26 15:05:53 dist-test-slave-7153 krb5kdc[23582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435153, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:05:53.779989 23952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:53.756805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:59039 (local address 127.9.123.126:39699)
0926 15:05:53.757353 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:53.757358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:53.759303 (+ 1945us) server_negotiation.cc:407] Connection header received
0926 15:05:53.761689 (+ 2386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:53.761694 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:53.761786 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:53.761892 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:53.764253 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:53.765317 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:53.768080 (+ 2763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:53.768617 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:53.768773 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:53.773082 (+ 4309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:53.773089 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:53.773094 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:53.773170 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:53.776097 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:53.777290 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:53.777296 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:53.777298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:53.777472 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:53.777999 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:53.778003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:53.778005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:53.778334 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:53.778489 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:53.779330 (+ 841us) server_negotiation.cc:299] Negotiation successful
0926 15:05:53.779600 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":327,"threads_started":1}
WARNING: no policy specified for kudu/127.9.123.67@KRBTEST.COM; defaulting to no policy
I20260926 15:05:53.781159 23953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:53.757779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.66:59039)
0926 15:05:53.758742 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:53.758801 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:53.761234 (+ 2433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:53.762229 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:53.762242 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:53.763063 (+ 821us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:05:53.763954 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:53.763986 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:53.765574 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:53.765581 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:53.767793 (+ 2212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:53.767817 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:53.768900 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:53.768910 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:53.769213 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:53.770085 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:05:53.770117 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:05:53.772756 (+ 2639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:05:53.776428 (+ 3672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:53.776437 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:53.776454 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:53.777044 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:53.777615 (+ 571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:53.777619 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:53.777621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:53.777860 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:53.778484 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:05:53.778494 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:05:53.778911 (+ 417us) client_negotiation.cc:769] Sending connection context
0926 15:05:53.779177 (+ 266us) client_negotiation.cc:240] Negotiation successful
0926 15:05:53.779807 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":448,"threads_started":1}
I20260926 15:05:53.783239 23950 heartbeater.cc:347] Connected to a master server at 127.9.123.126:39699
I20260926 15:05:53.783567 23950 heartbeater.cc:464] Registering TS with master...
I20260926 15:05:53.784339 23950 heartbeater.cc:511] Master 127.9.123.126:39699 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.67@KRBTEST.COM" created.
I20260926 15:05:53.786405 23627 ts_manager.cc:194] Registered new tserver with Master: cd61584d310b455ba36e80f6178ec37b (127.9.123.66:45037)
I20260926 15:05:53.787564 23627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.66@KRBTEST.COM'} at 127.9.123.66:59039
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
I20260926 15:05:53.819140 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:39699
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:44657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:05:54.165483 23956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:05:54.165796 23956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:05:54.165917 23956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:05:54.173605 23956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:05:54.173750 23956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:05:54.173918 23956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:05:54.183990 23956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:44657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:39699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:05:54.185387 23956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:05:54.186774 23956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:05:54.200413 23962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:54.200440 23964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:54.200667 23961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:54.201452 23956 server_base.cc:1034] running on GCE node
I20260926 15:05:54.202136 23956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:05:54.203246 23956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:05:54.204515 23956 hybrid_clock.cc:648] HybridClock initialized: now 1790435154204450 us; error 60 us; skew 500 ppm
I20260926 15:05:54.204900 23956 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:05:54 dist-test-slave-7153 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435154, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:54.208302 23956 init.cc:377] Logged in from keytab as kudu/127.9.123.67@KRBTEST.COM (short username kudu)
I20260926 15:05:54.209988 23956 webserver.cc:460] Webserver started at http://127.9.123.67:41789/ using document root <none> and password file <none>
I20260926 15:05:54.210741 23956 fs_manager.cc:359] Metadata directory not provided
I20260926 15:05:54.210865 23956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:05:54.211195 23956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:05:54.214377 23956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "a94aa4e153f14ee89870daa80e6b3f30"
format_stamp: "Formatted at 2026-09-26 15:05:54 on dist-test-slave-7153"
server_key: "5fa580af0e81ce462387b18dfc5f2317"
server_key_iv: "2fd45b928397cf224eb46a8602562593"
server_key_version: "encryptionkey@0"
I20260926 15:05:54.215205 23956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "a94aa4e153f14ee89870daa80e6b3f30"
format_stamp: "Formatted at 2026-09-26 15:05:54 on dist-test-slave-7153"
server_key: "5fa580af0e81ce462387b18dfc5f2317"
server_key_iv: "2fd45b928397cf224eb46a8602562593"
server_key_version: "encryptionkey@0"
I20260926 15:05:54.220988 23956 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260926 15:05:54.225369 23971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:54.226994 23956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260926 15:05:54.227180 23956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "a94aa4e153f14ee89870daa80e6b3f30"
format_stamp: "Formatted at 2026-09-26 15:05:54 on dist-test-slave-7153"
server_key: "5fa580af0e81ce462387b18dfc5f2317"
server_key_iv: "2fd45b928397cf224eb46a8602562593"
server_key_version: "encryptionkey@0"
I20260926 15:05:54.227422 23956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:05:54.252357 23956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:05:54.257306 23956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:05:54.257611 23956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:05:54.258649 23956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:05:54.260089 23956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:05:54.260174 23956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:54.260293 23956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:05:54.260344 23956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:54.265766 23816 heartbeater.cc:503] Master 127.9.123.126:39699 was elected leader, sending a full tablet report...
I20260926 15:05:54.295691 23956 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:33221
I20260926 15:05:54.295778 24083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:33221 every 8 connection(s)
I20260926 15:05:54.297091 23956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:05:54.303030 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 23956
I20260926 15:05:54.303226 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:05:54.303640 9709 external_mini_cluster.cc:1428] Setting key 758faa8524abe46c09ad9ba7d675093d
Sep 26 15:05:54 dist-test-slave-7153 krb5kdc[23582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435154, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:05:54.319417 24086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:54.299994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:48113 (local address 127.9.123.126:39699)
0926 15:05:54.300630 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:54.300637 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:54.301626 (+ 989us) server_negotiation.cc:407] Connection header received
0926 15:05:54.303669 (+ 2043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:54.303675 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:54.303788 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:54.303894 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:54.305937 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.306764 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.308480 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.308919 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.309026 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:54.312737 (+ 3711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:54.312743 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:54.312747 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:54.312788 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:54.315724 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:54.316649 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:54.316654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:54.316657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:54.316823 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:54.317349 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:54.317355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:54.317357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:54.317714 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:54.317889 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:54.318837 (+ 948us) server_negotiation.cc:299] Negotiation successful
0926 15:05:54.319088 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":367,"threads_started":1}
I20260926 15:05:54.320575 24087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:54.300640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.67:48113)
0926 15:05:54.301454 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:54.301498 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:54.303340 (+ 1842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:54.304215 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:54.304228 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:54.304922 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:05:54.305755 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:54.305779 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.306978 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.306984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:54.308321 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:54.308336 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.309148 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.309156 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:54.309384 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:54.310062 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:05:54.310091 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:05:54.312469 (+ 2378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:05:54.315918 (+ 3449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:54.315927 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:54.315944 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:54.316468 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:54.316922 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:54.316925 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:54.316927 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:54.317211 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:54.317870 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:05:54.317877 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:05:54.318391 (+ 514us) client_negotiation.cc:769] Sending connection context
0926 15:05:54.318694 (+ 303us) client_negotiation.cc:240] Negotiation successful
0926 15:05:54.319262 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":279,"threads_started":1}
I20260926 15:05:54.322247 24084 heartbeater.cc:347] Connected to a master server at 127.9.123.126:39699
I20260926 15:05:54.322595 24084 heartbeater.cc:464] Registering TS with master...
I20260926 15:05:54.323449 24084 heartbeater.cc:511] Master 127.9.123.126:39699 requested a full tablet report, sending...
I20260926 15:05:54.325078 23627 ts_manager.cc:194] Registered new tserver with Master: a94aa4e153f14ee89870daa80e6b3f30 (127.9.123.67:33221)
I20260926 15:05:54.325979 23627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.67@KRBTEST.COM'} at 127.9.123.67:48113
I20260926 15:05:54.332630 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 15:05:54.350498 24086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:54.337813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57908 (local address 127.9.123.126:39699)
0926 15:05:54.338006 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:54.338011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:54.338530 (+ 519us) server_negotiation.cc:407] Connection header received
0926 15:05:54.338657 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:54.338661 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:54.338757 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:54.338853 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:54.340211 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.340960 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.342414 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.342842 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.342947 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:54.345383 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:54.345389 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:54.345393 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:54.345431 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:54.347855 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:54.348658 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:54.348663 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:54.348665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:54.348807 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:54.349185 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:54.349190 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:54.349192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:54.349559 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:54.349759 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:54.350042 (+ 283us) server_negotiation.cc:299] Negotiation successful
0926 15:05:54.350262 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 15:05:54.354727 23627 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57908:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:05:54.361833 23627 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:05:54.398303 24100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:54.375427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35308 (local address 127.9.123.67:33221)
0926 15:05:54.387638 (+ 12211us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:54.387648 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:54.388924 (+ 1276us) server_negotiation.cc:407] Connection header received
0926 15:05:54.389037 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:54.389044 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:54.389321 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:54.389510 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:05:54.390483 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.392027 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.395122 (+ 3095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.396605 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.396741 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:54.396985 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:54.397804 (+ 819us) server_negotiation.cc:299] Negotiation successful
0926 15:05:54.397982 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12029,"thread_start_us":12227,"threads_started":1}
I20260926 15:05:54.398301 24099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:54.374796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:37823 (local address 127.0.0.1:60562)
0926 15:05:54.387457 (+ 12661us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:54.387526 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:54.387795 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:54.389433 (+ 1638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:54.389441 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:54.389475 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:05:54.389991 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:54.390009 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.392148 (+ 2139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.392153 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:54.394934 (+ 2781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:54.394959 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.397162 (+ 2203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.397170 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:54.397396 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:54.397463 (+ 67us) client_negotiation.cc:769] Sending connection context
0926 15:05:54.397592 (+ 129us) client_negotiation.cc:240] Negotiation successful
0926 15:05:54.397827 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12300,"mutex_wait_us":30,"thread_start_us":13535,"threads_started":1}
I20260926 15:05:54.398345 24098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:54.375119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:33221 (local address 127.0.0.1:35308)
0926 15:05:54.388642 (+ 13523us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:54.388661 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:54.388787 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:54.389816 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:54.389821 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:54.389851 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:05:54.390284 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:54.390297 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.392161 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.392165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:54.394958 (+ 2793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:54.394969 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.397433 (+ 2464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.397439 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:54.397630 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:54.397661 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 15:05:54.397749 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 15:05:54.397886 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13387,"mutex_wait_us":550,"thread_start_us":11799,"threads_started":1}
I20260926 15:05:54.399165 24097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:54.374846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60562 (local address 127.9.123.65:37823)
0926 15:05:54.388363 (+ 13517us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:54.388373 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:54.388409 (+ 36us) server_negotiation.cc:407] Connection header received
0926 15:05:54.388521 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:54.388529 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:54.388834 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:54.389052 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:05:54.390276 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.391966 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.395191 (+ 3225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.396587 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.396725 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:54.396980 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:54.398623 (+ 1643us) server_negotiation.cc:299] Negotiation successful
0926 15:05:54.398838 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13338,"thread_start_us":13153,"threads_started":1}
I20260926 15:05:54.403226 24102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:54.389037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46976 (local address 127.9.123.66:45037)
0926 15:05:54.392602 (+ 3565us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:54.392612 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:54.395318 (+ 2706us) server_negotiation.cc:407] Connection header received
0926 15:05:54.395433 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:54.395441 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:54.395771 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:54.395993 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:05:54.396725 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.398281 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.400708 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.402192 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.402338 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:54.402559 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:54.402774 (+ 215us) server_negotiation.cc:299] Negotiation successful
0926 15:05:54.402945 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3386,"thread_start_us":3277,"threads_started":1}
I20260926 15:05:54.403231 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:54.388766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:45037 (local address 127.0.0.1:46976)
0926 15:05:54.395004 (+ 6238us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:54.395022 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:54.395165 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:54.396016 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:54.396021 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:54.396054 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:05:54.396517 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:54.396533 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.398550 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.398556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:54.400520 (+ 1964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:54.400535 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.402414 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.402421 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:54.402592 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:54.402626 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 15:05:54.402710 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 15:05:54.402859 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6091,"thread_start_us":5238,"threads_started":1}
I20260926 15:05:54.403416 23751 tablet_service.cc:1467] Processing CreateTablet for tablet 20b97d3dc5714644ac2bd3fb477815c6 (DEFAULT_TABLE table=test-table [id=339fcc1f619c48bb99d2b9ec90f212ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:05:54.403416 24019 tablet_service.cc:1467] Processing CreateTablet for tablet 20b97d3dc5714644ac2bd3fb477815c6 (DEFAULT_TABLE table=test-table [id=339fcc1f619c48bb99d2b9ec90f212ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:05:54.405148 24019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20b97d3dc5714644ac2bd3fb477815c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:05:54.405942 23751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20b97d3dc5714644ac2bd3fb477815c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:05:54.407379 23885 tablet_service.cc:1467] Processing CreateTablet for tablet 20b97d3dc5714644ac2bd3fb477815c6 (DEFAULT_TABLE table=test-table [id=339fcc1f619c48bb99d2b9ec90f212ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:05:54.408944 23885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20b97d3dc5714644ac2bd3fb477815c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:05:54.418043 24103 tablet_bootstrap.cc:492] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc: Bootstrap starting.
I20260926 15:05:54.421119 24104 tablet_bootstrap.cc:492] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30: Bootstrap starting.
I20260926 15:05:54.424244 24105 tablet_bootstrap.cc:492] T 20b97d3dc5714644ac2bd3fb477815c6 P cd61584d310b455ba36e80f6178ec37b: Bootstrap starting.
I20260926 15:05:54.424319 24103 tablet_bootstrap.cc:654] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc: Neither blocks nor log segments found. Creating new log.
I20260926 15:05:54.424793 24104 tablet_bootstrap.cc:654] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30: Neither blocks nor log segments found. Creating new log.
I20260926 15:05:54.425828 24103 log.cc:824] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc: Log is configured to *not* fsync() on all Append() calls
I20260926 15:05:54.426082 24104 log.cc:824] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30: Log is configured to *not* fsync() on all Append() calls
I20260926 15:05:54.427799 24105 tablet_bootstrap.cc:654] T 20b97d3dc5714644ac2bd3fb477815c6 P cd61584d310b455ba36e80f6178ec37b: Neither blocks nor log segments found. Creating new log.
I20260926 15:05:54.429136 24103 tablet_bootstrap.cc:492] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc: No bootstrap required, opened a new log
I20260926 15:05:54.429145 24105 log.cc:824] T 20b97d3dc5714644ac2bd3fb477815c6 P cd61584d310b455ba36e80f6178ec37b: Log is configured to *not* fsync() on all Append() calls
I20260926 15:05:54.429359 24104 tablet_bootstrap.cc:492] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30: No bootstrap required, opened a new log
I20260926 15:05:54.429440 24103 ts_tablet_manager.cc:1397] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260926 15:05:54.429689 24104 ts_tablet_manager.cc:1397] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260926 15:05:54.434114 24105 tablet_bootstrap.cc:492] T 20b97d3dc5714644ac2bd3fb477815c6 P cd61584d310b455ba36e80f6178ec37b: No bootstrap required, opened a new log
I20260926 15:05:54.434464 24105 ts_tablet_manager.cc:1397] T 20b97d3dc5714644ac2bd3fb477815c6 P cd61584d310b455ba36e80f6178ec37b: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260926 15:05:54.437999 24103 raft_consensus.cc:348] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a122d72de34473b78cd74c6086d1dc" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 37823 } } peers { permanent_uuid: "cd61584d310b455ba36e80f6178ec37b" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45037 } } peers { permanent_uuid: "a94aa4e153f14ee89870daa80e6b3f30" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33221 } }
I20260926 15:05:54.438405 24103 raft_consensus.cc:374] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:05:54.438524 24103 raft_consensus.cc:729] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4a122d72de34473b78cd74c6086d1dc, State: Initialized, Role: FOLLOWER
I20260926 15:05:54.439152 24103 consensus_queue.cc:260] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a122d72de34473b78cd74c6086d1dc" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 37823 } } peers { permanent_uuid: "cd61584d310b455ba36e80f6178ec37b" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45037 } } peers { permanent_uuid: "a94aa4e153f14ee89870daa80e6b3f30" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33221 } }
I20260926 15:05:54.440173 24104 raft_consensus.cc:348] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a122d72de34473b78cd74c6086d1dc" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 37823 } } peers { permanent_uuid: "cd61584d310b455ba36e80f6178ec37b" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45037 } } peers { permanent_uuid: "a94aa4e153f14ee89870daa80e6b3f30" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33221 } }
I20260926 15:05:54.440702 24104 raft_consensus.cc:374] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:05:54.440841 24104 raft_consensus.cc:729] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a94aa4e153f14ee89870daa80e6b3f30, State: Initialized, Role: FOLLOWER
I20260926 15:05:54.441026 24103 ts_tablet_manager.cc:1428] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260926 15:05:54.441692 24104 consensus_queue.cc:260] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a122d72de34473b78cd74c6086d1dc" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 37823 } } peers { permanent_uuid: "cd61584d310b455ba36e80f6178ec37b" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45037 } } peers { permanent_uuid: "a94aa4e153f14ee89870daa80e6b3f30" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33221 } }
I20260926 15:05:54.443166 24084 heartbeater.cc:503] Master 127.9.123.126:39699 was elected leader, sending a full tablet report...
I20260926 15:05:54.443908 24104 ts_tablet_manager.cc:1428] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260926 15:05:54.444885 24105 raft_consensus.cc:348] T 20b97d3dc5714644ac2bd3fb477815c6 P cd61584d310b455ba36e80f6178ec37b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a122d72de34473b78cd74c6086d1dc" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 37823 } } peers { permanent_uuid: "cd61584d310b455ba36e80f6178ec37b" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45037 } } peers { permanent_uuid: "a94aa4e153f14ee89870daa80e6b3f30" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33221 } }
I20260926 15:05:54.445367 24105 raft_consensus.cc:374] T 20b97d3dc5714644ac2bd3fb477815c6 P cd61584d310b455ba36e80f6178ec37b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:05:54.445466 24105 raft_consensus.cc:729] T 20b97d3dc5714644ac2bd3fb477815c6 P cd61584d310b455ba36e80f6178ec37b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd61584d310b455ba36e80f6178ec37b, State: Initialized, Role: FOLLOWER
I20260926 15:05:54.446122 24105 consensus_queue.cc:260] T 20b97d3dc5714644ac2bd3fb477815c6 P cd61584d310b455ba36e80f6178ec37b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a122d72de34473b78cd74c6086d1dc" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 37823 } } peers { permanent_uuid: "cd61584d310b455ba36e80f6178ec37b" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45037 } } peers { permanent_uuid: "a94aa4e153f14ee89870daa80e6b3f30" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33221 } }
I20260926 15:05:54.446786 24110 raft_consensus.cc:482] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:05:54.447050 24110 raft_consensus.cc:504] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a122d72de34473b78cd74c6086d1dc" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 37823 } } peers { permanent_uuid: "cd61584d310b455ba36e80f6178ec37b" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45037 } } peers { permanent_uuid: "a94aa4e153f14ee89870daa80e6b3f30" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33221 } }
I20260926 15:05:54.448678 24110 leader_election.cc:288] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4a122d72de34473b78cd74c6086d1dc (127.9.123.65:37823), cd61584d310b455ba36e80f6178ec37b (127.9.123.66:45037)
I20260926 15:05:54.452999 23950 heartbeater.cc:503] Master 127.9.123.126:39699 was elected leader, sending a full tablet report...
I20260926 15:05:54.453787 24105 ts_tablet_manager.cc:1428] T 20b97d3dc5714644ac2bd3fb477815c6 P cd61584d310b455ba36e80f6178ec37b: Time spent starting tablet: real 0.019s user 0.012s sys 0.008s
I20260926 15:05:54.455456 24109 raft_consensus.cc:482] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:05:54.455813 24109 raft_consensus.cc:504] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a122d72de34473b78cd74c6086d1dc" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 37823 } } peers { permanent_uuid: "cd61584d310b455ba36e80f6178ec37b" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45037 } } peers { permanent_uuid: "a94aa4e153f14ee89870daa80e6b3f30" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33221 } }
I20260926 15:05:54.456645 24087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:54.449036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:37823 (local address 127.9.123.67:42757)
0926 15:05:54.449697 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:54.449719 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:54.449850 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:54.450401 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:54.450407 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:54.450435 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:05:54.450664 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:54.450675 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.452169 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.452176 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:54.454137 (+ 1961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:54.454155 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.455909 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.455915 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:54.456100 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:54.456134 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 15:05:54.456222 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 15:05:54.456363 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522}
I20260926 15:05:54.457226 24097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:54.449217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:42757 (local address 127.9.123.65:37823)
0926 15:05:54.449406 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:54.449412 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:54.449967 (+ 555us) server_negotiation.cc:407] Connection header received
0926 15:05:54.450065 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:54.450070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:54.450148 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:54.450226 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:05:54.450829 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.452006 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.454348 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.455568 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.455743 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:54.455824 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:54.456827 (+ 1003us) server_negotiation.cc:299] Negotiation successful
0926 15:05:54.456968 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 15:05:54.457978 24109 leader_election.cc:288] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd61584d310b455ba36e80f6178ec37b (127.9.123.66:45037), a94aa4e153f14ee89870daa80e6b3f30 (127.9.123.67:33221)
I20260926 15:05:54.460382 23771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20b97d3dc5714644ac2bd3fb477815c6" candidate_uuid: "a94aa4e153f14ee89870daa80e6b3f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4a122d72de34473b78cd74c6086d1dc" is_pre_election: true
I20260926 15:05:54.460894 23771 raft_consensus.cc:2454] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a94aa4e153f14ee89870daa80e6b3f30 in term 0.
I20260926 15:05:54.461685 23973 leader_election.cc:302] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a94aa4e153f14ee89870daa80e6b3f30, b4a122d72de34473b78cd74c6086d1dc; no voters:
I20260926 15:05:54.462167 24102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:54.449680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:42765 (local address 127.9.123.66:45037)
0926 15:05:54.449890 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:54.449896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:54.454796 (+ 4900us) server_negotiation.cc:407] Connection header received
0926 15:05:54.454945 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:54.454950 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:54.455024 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:54.455116 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:05:54.455951 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.457001 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.459441 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.460731 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.460869 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:54.460943 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:54.461697 (+ 754us) server_negotiation.cc:299] Negotiation successful
0926 15:05:54.461856 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 15:05:54.462270 24110 raft_consensus.cc:2781] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:05:54.462420 24110 raft_consensus.cc:482] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:05:54.462555 24110 raft_consensus.cc:3037] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:05:54.463186 24112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:54.449460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:45037 (local address 127.9.123.67:42765)
0926 15:05:54.454680 (+ 5220us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:54.454700 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:54.454836 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:54.455372 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:54.455377 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:54.455407 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:05:54.455773 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:54.455786 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.457231 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.457236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:54.459245 (+ 2009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:54.459262 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.461153 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.461158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:54.461486 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:54.461527 (+ 41us) client_negotiation.cc:769] Sending connection context
0926 15:05:54.462601 (+ 1074us) client_negotiation.cc:240] Negotiation successful
0926 15:05:54.462780 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5090,"thread_start_us":4923,"threads_started":1}
I20260926 15:05:54.464421 23905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20b97d3dc5714644ac2bd3fb477815c6" candidate_uuid: "a94aa4e153f14ee89870daa80e6b3f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd61584d310b455ba36e80f6178ec37b" is_pre_election: true
I20260926 15:05:54.465086 23905 raft_consensus.cc:2454] T 20b97d3dc5714644ac2bd3fb477815c6 P cd61584d310b455ba36e80f6178ec37b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a94aa4e153f14ee89870daa80e6b3f30 in term 0.
I20260926 15:05:54.466573 24110 raft_consensus.cc:504] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a122d72de34473b78cd74c6086d1dc" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 37823 } } peers { permanent_uuid: "cd61584d310b455ba36e80f6178ec37b" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45037 } } peers { permanent_uuid: "a94aa4e153f14ee89870daa80e6b3f30" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33221 } }
I20260926 15:05:54.467449 24110 leader_election.cc:288] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30 [CANDIDATE]: Term 1 election: Requested vote from peers b4a122d72de34473b78cd74c6086d1dc (127.9.123.65:37823), cd61584d310b455ba36e80f6178ec37b (127.9.123.66:45037)
I20260926 15:05:54.468058 23905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20b97d3dc5714644ac2bd3fb477815c6" candidate_uuid: "a94aa4e153f14ee89870daa80e6b3f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd61584d310b455ba36e80f6178ec37b"
I20260926 15:05:54.468295 23905 raft_consensus.cc:3037] T 20b97d3dc5714644ac2bd3fb477815c6 P cd61584d310b455ba36e80f6178ec37b [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:05:54.469034 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:54.458832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:33959 (local address 127.9.123.66:45037)
0926 15:05:54.461639 (+ 2807us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:54.461644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:54.461667 (+ 23us) server_negotiation.cc:407] Connection header received
0926 15:05:54.461722 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:54.461727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:54.461793 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:54.462516 (+ 723us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:05:54.462795 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.463927 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.466682 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.467932 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.468221 (+ 289us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:54.468299 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:54.468561 (+ 262us) server_negotiation.cc:299] Negotiation successful
0926 15:05:54.468707 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2672,"thread_start_us":2427,"threads_started":1}
I20260926 15:05:54.469079 24115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:54.458302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:45037 (local address 127.9.123.65:33959)
0926 15:05:54.460600 (+ 2298us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:54.460620 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:54.460745 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:54.461935 (+ 1190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:54.461940 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:54.461967 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:05:54.462359 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:54.462374 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.464082 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.464087 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:54.466447 (+ 2360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:54.466466 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.468195 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.468201 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:54.468401 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:54.468438 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 15:05:54.468537 (+ 99us) client_negotiation.cc:240] Negotiation successful
0926 15:05:54.468697 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1105,"thread_start_us":7683,"threads_started":1}
I20260926 15:05:54.470086 23904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20b97d3dc5714644ac2bd3fb477815c6" candidate_uuid: "b4a122d72de34473b78cd74c6086d1dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd61584d310b455ba36e80f6178ec37b" is_pre_election: true
I20260926 15:05:54.471222 23905 raft_consensus.cc:2454] T 20b97d3dc5714644ac2bd3fb477815c6 P cd61584d310b455ba36e80f6178ec37b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a94aa4e153f14ee89870daa80e6b3f30 in term 1.
I20260926 15:05:54.471648 23973 leader_election.cc:302] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a94aa4e153f14ee89870daa80e6b3f30, cd61584d310b455ba36e80f6178ec37b; no voters:
I20260926 15:05:54.471683 23771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20b97d3dc5714644ac2bd3fb477815c6" candidate_uuid: "a94aa4e153f14ee89870daa80e6b3f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4a122d72de34473b78cd74c6086d1dc"
I20260926 15:05:54.471896 23771 raft_consensus.cc:3037] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:05:54.472729 24100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:54.458896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:38233 (local address 127.9.123.67:33221)
0926 15:05:54.459210 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:54.459215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:54.466434 (+ 7219us) server_negotiation.cc:407] Connection header received
0926 15:05:54.466615 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:54.466618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:54.466675 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:54.466745 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:05:54.467239 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.468247 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.470529 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.471701 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.471821 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:54.471887 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:54.472367 (+ 480us) server_negotiation.cc:299] Negotiation successful
0926 15:05:54.472501 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161}
I20260926 15:05:54.472829 24113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:54.458842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:33221 (local address 127.9.123.65:38233)
0926 15:05:54.466322 (+ 7480us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:54.466340 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:54.466481 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:54.466783 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:54.466787 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:54.466813 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:05:54.467086 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:54.467095 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.468438 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.468443 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:54.470341 (+ 1898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:54.470357 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.472079 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.472085 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:54.472220 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:54.472243 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 15:05:54.472329 (+ 86us) client_negotiation.cc:240] Negotiation successful
0926 15:05:54.472476 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7339,"thread_start_us":642,"threads_started":1}
I20260926 15:05:54.473734 24039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20b97d3dc5714644ac2bd3fb477815c6" candidate_uuid: "b4a122d72de34473b78cd74c6086d1dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a94aa4e153f14ee89870daa80e6b3f30" is_pre_election: true
I20260926 15:05:54.474193 24039 raft_consensus.cc:2379] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b4a122d72de34473b78cd74c6086d1dc in current term 1: Already voted for candidate a94aa4e153f14ee89870daa80e6b3f30 in this term.
I20260926 15:05:54.474521 23771 raft_consensus.cc:2454] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a94aa4e153f14ee89870daa80e6b3f30 in term 1.
I20260926 15:05:54.474823 24110 raft_consensus.cc:2781] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:05:54.474881 23706 leader_election.cc:302] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b4a122d72de34473b78cd74c6086d1dc; no voters: a94aa4e153f14ee89870daa80e6b3f30, cd61584d310b455ba36e80f6178ec37b
I20260926 15:05:54.475312 24109 raft_consensus.cc:2726] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 15:05:54.475546 24110 raft_consensus.cc:686] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30 [term 1 LEADER]: Becoming Leader. State: Replica: a94aa4e153f14ee89870daa80e6b3f30, State: Running, Role: LEADER
I20260926 15:05:54.476147 24110 consensus_queue.cc:237] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a122d72de34473b78cd74c6086d1dc" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 37823 } } peers { permanent_uuid: "cd61584d310b455ba36e80f6178ec37b" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45037 } } peers { permanent_uuid: "a94aa4e153f14ee89870daa80e6b3f30" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33221 } }
I20260926 15:05:54.480444 23627 catalog_manager.cc:5495] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30 reported cstate change: term changed from 0 to 1, leader changed from <none> to a94aa4e153f14ee89870daa80e6b3f30 (127.9.123.67). New cstate: current_term: 1 leader_uuid: "a94aa4e153f14ee89870daa80e6b3f30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4a122d72de34473b78cd74c6086d1dc" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 37823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd61584d310b455ba36e80f6178ec37b" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a94aa4e153f14ee89870daa80e6b3f30" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33221 } health_report { overall_health: HEALTHY } } }
W20260926 15:05:54.484330 23817 tablet.cc:2369] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 15:05:54.505756 23951 tablet.cc:2369] T 20b97d3dc5714644ac2bd3fb477815c6 P cd61584d310b455ba36e80f6178ec37b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:05:54.510087 24100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:54.504345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35312 (local address 127.9.123.67:33221)
0926 15:05:54.504597 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:54.504603 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:54.504625 (+ 22us) server_negotiation.cc:407] Connection header received
0926 15:05:54.504730 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:54.504734 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:54.504862 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:54.505007 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:05:54.505476 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.506327 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.507773 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.508189 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.508307 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:54.508863 (+ 556us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:05:54.509442 (+ 579us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:05:54.509586 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:54.509687 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 15:05:54.509777 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260926 15:05:54.518906 23905 raft_consensus.cc:1261] T 20b97d3dc5714644ac2bd3fb477815c6 P cd61584d310b455ba36e80f6178ec37b [term 1 FOLLOWER]: Refusing update from remote peer a94aa4e153f14ee89870daa80e6b3f30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:05:54.518975 23771 raft_consensus.cc:1261] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc [term 1 FOLLOWER]: Refusing update from remote peer a94aa4e153f14ee89870daa80e6b3f30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:05:54.519825 24110 consensus_queue.cc:1035] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd61584d310b455ba36e80f6178ec37b" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:05:54.520153 24116 consensus_queue.cc:1035] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4a122d72de34473b78cd74c6086d1dc" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 37823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:05:54.548563 23626 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:05:54.548877 23626 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:05:54.552950 23626 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b: Sending DeleteTablet for 3 replicas of tablet 20b97d3dc5714644ac2bd3fb477815c6
I20260926 15:05:54.554383 23885 tablet_service.cc:1514] Processing DeleteTablet for tablet 20b97d3dc5714644ac2bd3fb477815c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:05:54 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:46976
I20260926 15:05:54.554395 23751 tablet_service.cc:1514] Processing DeleteTablet for tablet 20b97d3dc5714644ac2bd3fb477815c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:05:54 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:60562
I20260926 15:05:54.554551 24019 tablet_service.cc:1514] Processing DeleteTablet for tablet 20b97d3dc5714644ac2bd3fb477815c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:05:54 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:35308
I20260926 15:05:54.555611 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 23688
I20260926 15:05:54.566895 24130 tablet_replica.cc:330] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc: stopping tablet replica
I20260926 15:05:54.567579 24129 tablet_replica.cc:330] T 20b97d3dc5714644ac2bd3fb477815c6 P cd61584d310b455ba36e80f6178ec37b: stopping tablet replica
I20260926 15:05:54.568104 24129 raft_consensus.cc:2229] T 20b97d3dc5714644ac2bd3fb477815c6 P cd61584d310b455ba36e80f6178ec37b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:05:54.568691 24129 raft_consensus.cc:2258] T 20b97d3dc5714644ac2bd3fb477815c6 P cd61584d310b455ba36e80f6178ec37b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:05:54.569572 24130 raft_consensus.cc:2229] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:05:54.570104 24130 raft_consensus.cc:2258] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:05:54.572233 24130 ts_tablet_manager.cc:1905] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:05:54.572777 24129 ts_tablet_manager.cc:1905] T 20b97d3dc5714644ac2bd3fb477815c6 P cd61584d310b455ba36e80f6178ec37b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:05:54.579936 24130 ts_tablet_manager.cc:1918] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:05:54.579934 24129 ts_tablet_manager.cc:1918] T 20b97d3dc5714644ac2bd3fb477815c6 P cd61584d310b455ba36e80f6178ec37b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:05:54.580147 24129 log.cc:1196] T 20b97d3dc5714644ac2bd3fb477815c6 P cd61584d310b455ba36e80f6178ec37b: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/20b97d3dc5714644ac2bd3fb477815c6
I20260926 15:05:54.580355 24130 log.cc:1196] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/wal/wals/20b97d3dc5714644ac2bd3fb477815c6
I20260926 15:05:54.580669 24129 ts_tablet_manager.cc:1939] T 20b97d3dc5714644ac2bd3fb477815c6 P cd61584d310b455ba36e80f6178ec37b: Deleting consensus metadata
I20260926 15:05:54.581035 24130 ts_tablet_manager.cc:1939] T 20b97d3dc5714644ac2bd3fb477815c6 P b4a122d72de34473b78cd74c6086d1dc: Deleting consensus metadata
I20260926 15:05:54.582712 23612 catalog_manager.cc:4841] TS cd61584d310b455ba36e80f6178ec37b (127.9.123.66:45037): tablet 20b97d3dc5714644ac2bd3fb477815c6 (table test-table [id=339fcc1f619c48bb99d2b9ec90f212ad]) successfully deleted
I20260926 15:05:54.583063 23612 catalog_manager.cc:4841] TS b4a122d72de34473b78cd74c6086d1dc (127.9.123.65:37823): tablet 20b97d3dc5714644ac2bd3fb477815c6 (table test-table [id=339fcc1f619c48bb99d2b9ec90f212ad]) successfully deleted
I20260926 15:05:54.586467 24131 tablet_replica.cc:330] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30: stopping tablet replica
I20260926 15:05:54.586994 24131 raft_consensus.cc:2229] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30 [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:05:54.587836 24131 raft_consensus.cc:2258] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:05:54.589921 24131 ts_tablet_manager.cc:1905] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 26 15:05:54 dist-test-slave-7153 krb5kdc[23582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.65@KRBTEST.COM
I20260926 15:05:54.598168 24131 ts_tablet_manager.cc:1918] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:05:54.598384 24131 log.cc:1196] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/20b97d3dc5714644ac2bd3fb477815c6
I20260926 15:05:54.598971 24131 ts_tablet_manager.cc:1939] T 20b97d3dc5714644ac2bd3fb477815c6 P a94aa4e153f14ee89870daa80e6b3f30: Deleting consensus metadata
I20260926 15:05:54.600996 23613 catalog_manager.cc:4841] TS a94aa4e153f14ee89870daa80e6b3f30 (127.9.123.67:33221): tablet 20b97d3dc5714644ac2bd3fb477815c6 (table test-table [id=339fcc1f619c48bb99d2b9ec90f212ad]) successfully deleted
I20260926 15:05:54.604974 24097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:54.556901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60570 (local address 127.9.123.65:37823)
0926 15:05:54.557129 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:54.557137 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:54.585764 (+ 28627us) server_negotiation.cc:407] Connection header received
0926 15:05:54.585965 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:54.585971 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:54.586055 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:54.586172 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:54.588997 (+ 2825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.590165 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.592317 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.592979 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.593077 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:54.598702 (+ 5625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:54.598719 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:54.598735 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:54.598779 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:54.601987 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:54.602747 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:54.602756 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:54.602758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:54.602898 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:54.603306 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:54.603310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:54.603312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:54.603723 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:54.604015 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:54.604372 (+ 357us) server_negotiation.cc:299] Negotiation successful
0926 15:05:54.604608 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 15:05:54.605759 23811 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:05:54.699105 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 23688
I20260926 15:05:54.715000 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 23822
Sep 26 15:05:54 dist-test-slave-7153 krb5kdc[23582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.66@KRBTEST.COM
I20260926 15:05:54.732191 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:54.715865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46992 (local address 127.9.123.66:45037)
0926 15:05:54.716351 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:54.716356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:54.716371 (+ 15us) server_negotiation.cc:407] Connection header received
0926 15:05:54.716423 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:54.716427 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:54.716495 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:54.716580 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:54.717933 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.718679 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.720082 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.720478 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.720588 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:54.725568 (+ 4980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:54.725586 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:54.725603 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:54.725680 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:54.728830 (+ 3150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:54.730076 (+ 1246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:54.730083 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:54.730086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:54.730289 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:54.730684 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:54.730687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:54.730689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:54.731078 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:54.731332 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:54.731623 (+ 291us) server_negotiation.cc:299] Negotiation successful
0926 15:05:54.731882 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358}
I20260926 15:05:54.732933 23945 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:05:54.829797 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 23822
I20260926 15:05:54.844949 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 23956
Sep 26 15:05:54 dist-test-slave-7153 krb5kdc[23582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.67@KRBTEST.COM
I20260926 15:05:54.864611 24100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:54.845768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35322 (local address 127.9.123.67:33221)
0926 15:05:54.845973 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:54.845977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:54.845999 (+ 22us) server_negotiation.cc:407] Connection header received
0926 15:05:54.846222 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:54.846230 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:54.846296 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:54.846397 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:54.848310 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.849077 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.851251 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:54.851782 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:54.851894 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:54.858085 (+ 6191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:54.858106 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:54.858124 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:54.858187 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:54.861372 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:54.862397 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:54.862408 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:54.862410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:54.862568 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:54.863020 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:54.863023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:54.863025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:54.863469 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:54.863709 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:54.864104 (+ 395us) server_negotiation.cc:299] Negotiation successful
0926 15:05:54.864325 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 15:05:54.865396 24079 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:05:54.989270 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 23956
I20260926 15:05:55.005265 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 23595
I20260926 15:05:55.006574 23657 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:05:55.120895 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 23595
I20260926 15:05:55.138801 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:39699
--webserver_interface=127.9.123.126
--webserver_port=34565
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:44657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:39699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:05:55.474659 24137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:05:55.474965 24137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:05:55.475085 24137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:05:55.482707 24137 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:05:55.482838 24137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:05:55.482897 24137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:05:55.482975 24137 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:05:55.483027 24137 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:05:55.493554 24137 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:44657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:39699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:39699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=34565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:05:55.494966 24137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:05:55.496383 24137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:05:55.519048 24146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:55.519080 24143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:55.519007 24144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:55.519860 24137 server_base.cc:1034] running on GCE node
I20260926 15:05:55.520871 24137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:05:55.522720 24137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:05:55.523976 24137 hybrid_clock.cc:648] HybridClock initialized: now 1790435155523857 us; error 112 us; skew 500 ppm
I20260926 15:05:55.524502 24137 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:05:55 dist-test-slave-7153 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435155, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:55.529915 24137 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:05:55.531734 24137 webserver.cc:460] Webserver started at http://127.9.123.126:34565/ using document root <none> and password file <none>
I20260926 15:05:55.532568 24137 fs_manager.cc:359] Metadata directory not provided
I20260926 15:05:55.532701 24137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:05:55.539381 24137 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 15:05:55.542655 24153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:55.544164 24137 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:05:55.544368 24137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "f1b5473193be4bba9ed73a4ef5eae05b"
format_stamp: "Formatted at 2026-09-26 15:05:52 on dist-test-slave-7153"
server_key: "ef60863b8c1e4db4b0c7973ace27d81e"
server_key_iv: "23aaed99722434801033d129cbbaee7d"
server_key_version: "encryptionkey@0"
I20260926 15:05:55.545543 24137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:05:55.572340 24137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:05:55.576998 24137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:05:55.577310 24137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:05:55.595254 24137 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:39699
I20260926 15:05:55.595279 24204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:39699 every 8 connection(s)
I20260926 15:05:55.596946 24137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:05:55.601857 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 24137
I20260926 15:05:55.602932 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:37823
--local_ip_for_outbound_sockets=127.9.123.65
--tserver_master_addrs=127.9.123.126:39699
--webserver_port=37973
--webserver_interface=127.9.123.65
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:44657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:05:55.606038 24205 sys_catalog.cc:263] Verifying existing consensus state
I20260926 15:05:55.611135 24205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b: Bootstrap starting.
I20260926 15:05:55.631799 24205 log.cc:824] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b: Log is configured to *not* fsync() on all Append() calls
I20260926 15:05:55.645982 24205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 15:05:55.646530 24205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b: Bootstrap complete.
I20260926 15:05:55.656188 24205 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b5473193be4bba9ed73a4ef5eae05b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39699 } }
I20260926 15:05:55.657168 24205 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1b5473193be4bba9ed73a4ef5eae05b, State: Initialized, Role: FOLLOWER
I20260926 15:05:55.657840 24205 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b5473193be4bba9ed73a4ef5eae05b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39699 } }
I20260926 15:05:55.658084 24205 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:05:55.658202 24205 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:05:55.658361 24205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 1 FOLLOWER]: Advancing to term 2
I20260926 15:05:55.662386 24205 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b5473193be4bba9ed73a4ef5eae05b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39699 } }
I20260926 15:05:55.662942 24205 leader_election.cc:302] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1b5473193be4bba9ed73a4ef5eae05b; no voters:
I20260926 15:05:55.663549 24205 leader_election.cc:288] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 15:05:55.663688 24209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 2 FOLLOWER]: Leader election won for term 2
I20260926 15:05:55.664798 24209 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 2 LEADER]: Becoming Leader. State: Replica: f1b5473193be4bba9ed73a4ef5eae05b, State: Running, Role: LEADER
I20260926 15:05:55.665411 24209 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b5473193be4bba9ed73a4ef5eae05b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39699 } }
I20260926 15:05:55.665874 24205 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:05:55.667826 24211 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1b5473193be4bba9ed73a4ef5eae05b. Latest consensus state: current_term: 2 leader_uuid: "f1b5473193be4bba9ed73a4ef5eae05b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b5473193be4bba9ed73a4ef5eae05b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39699 } } }
I20260926 15:05:55.668215 24211 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [sys.catalog]: This master's current role is: LEADER
I20260926 15:05:55.668808 24210 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f1b5473193be4bba9ed73a4ef5eae05b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b5473193be4bba9ed73a4ef5eae05b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39699 } } }
I20260926 15:05:55.669127 24210 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [sys.catalog]: This master's current role is: LEADER
I20260926 15:05:55.679709 24215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:05:55.688881 24215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:05:55.690531 24215 catalog_manager.cc:1259] Loaded cluster ID: 07a465c2cd634c1e8cc51a2cb3135d52
I20260926 15:05:55.690755 24215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:05:55.694137 24215 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260926 15:05:55.711702 24215 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260926 15:05:55.722205 24215 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:05:55.724210 24215 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b: Loaded TSK: 0
I20260926 15:05:55.726086 24215 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 15:05:55.985033 24207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:05:55.985507 24207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:05:55.985666 24207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:05:55.994850 24207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:05:55.994997 24207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:05:55.995158 24207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:05:56.005791 24207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:44657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:37823
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=37973
--tserver_master_addrs=127.9.123.126:39699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:05:56.007202 24207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:05:56.008638 24207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:05:56.025290 24233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:56.025305 24232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:56.026779 24235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:56.030222 24207 server_base.cc:1034] running on GCE node
I20260926 15:05:56.031066 24207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:05:56.032405 24207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:05:56.033684 24207 hybrid_clock.cc:648] HybridClock initialized: now 1790435156033616 us; error 57 us; skew 500 ppm
I20260926 15:05:56.034111 24207 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:05:56 dist-test-slave-7153 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435156, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:56.044657 24207 init.cc:377] Logged in from keytab as kudu/127.9.123.65@KRBTEST.COM (short username kudu)
I20260926 15:05:56.046644 24207 webserver.cc:460] Webserver started at http://127.9.123.65:37973/ using document root <none> and password file <none>
I20260926 15:05:56.047605 24207 fs_manager.cc:359] Metadata directory not provided
I20260926 15:05:56.047765 24207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:05:56.055121 24207 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 15:05:56.058601 24242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:56.060369 24207 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:05:56.060563 24207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "b4a122d72de34473b78cd74c6086d1dc"
format_stamp: "Formatted at 2026-09-26 15:05:53 on dist-test-slave-7153"
server_key: "44fa0d2ab79b8b5a5f35ff3e7ba3e1fc"
server_key_iv: "03c41bc14fe4f82f0827b1b60b7cfd43"
server_key_version: "encryptionkey@0"
I20260926 15:05:56.062094 24207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:05:56.077868 24207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:05:56.084039 24207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:05:56.084349 24207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:05:56.085638 24207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:05:56.087357 24207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:05:56.087474 24207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:56.087591 24207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:05:56.087659 24207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:56.120788 24207 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:37823
I20260926 15:05:56.120875 24354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:37823 every 8 connection(s)
I20260926 15:05:56.122368 24207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:05:56.128124 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 24207
I20260926 15:05:56.128782 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:45037
--local_ip_for_outbound_sockets=127.9.123.66
--tserver_master_addrs=127.9.123.126:39699
--webserver_port=42239
--webserver_interface=127.9.123.66
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:44657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 15:05:56 dist-test-slave-7153 krb5kdc[23582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435156, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:05:56.147707 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:56.125393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:47075 (local address 127.9.123.126:39699)
0926 15:05:56.126038 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:56.126051 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:56.127077 (+ 1026us) server_negotiation.cc:407] Connection header received
0926 15:05:56.129147 (+ 2070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:56.129169 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:56.129585 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:56.129984 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:56.131771 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:56.133162 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:56.134876 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:56.135564 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:56.135769 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:56.139726 (+ 3957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:56.139736 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:56.139754 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:56.139797 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:56.142766 (+ 2969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:56.143831 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:56.143839 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:56.143845 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:56.144024 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:56.144559 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:56.144563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:56.144565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:56.145078 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:56.145291 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:56.146331 (+ 1040us) server_negotiation.cc:299] Negotiation successful
0926 15:05:56.146699 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":283,"threads_started":1}
I20260926 15:05:56.148288 24358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:56.125636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.65:47075)
0926 15:05:56.126500 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:56.126558 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:56.128461 (+ 1903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:56.130230 (+ 1769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:56.130243 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:56.130887 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:05:56.131507 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:56.131528 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:56.133344 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:56.133349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:56.134661 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:56.134679 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:56.135832 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:56.135840 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:56.136150 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:56.136840 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:05:56.136865 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:05:56.139379 (+ 2514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:05:56.142953 (+ 3574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:56.142963 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:56.142981 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:56.143662 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:56.144163 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:56.144168 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:56.144172 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:56.144445 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:56.145273 (+ 828us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:05:56.145282 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:05:56.145754 (+ 472us) client_negotiation.cc:769] Sending connection context
0926 15:05:56.146106 (+ 352us) client_negotiation.cc:240] Negotiation successful
0926 15:05:56.146727 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":331,"threads_started":1}
I20260926 15:05:56.152187 24355 heartbeater.cc:347] Connected to a master server at 127.9.123.126:39699
I20260926 15:05:56.152577 24355 heartbeater.cc:464] Registering TS with master...
I20260926 15:05:56.153581 24355 heartbeater.cc:511] Master 127.9.123.126:39699 requested a full tablet report, sending...
I20260926 15:05:56.156260 24170 ts_manager.cc:194] Registered new tserver with Master: b4a122d72de34473b78cd74c6086d1dc (127.9.123.65:37823)
I20260926 15:05:56.157876 24170 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.65@KRBTEST.COM'} at 127.9.123.65:47075
W20260926 15:05:56.472795 24359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:05:56.473124 24359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:05:56.473258 24359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:05:56.480844 24359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:05:56.480957 24359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:05:56.481149 24359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:05:56.491508 24359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:44657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:45037
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=42239
--tserver_master_addrs=127.9.123.126:39699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:05:56.493093 24359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:05:56.494576 24359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:05:56.508451 24364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:56.508412 24365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:56.508430 24367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:56.514976 24359 server_base.cc:1034] running on GCE node
I20260926 15:05:56.515715 24359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:05:56.516744 24359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:05:56.517979 24359 hybrid_clock.cc:648] HybridClock initialized: now 1790435156517912 us; error 56 us; skew 500 ppm
I20260926 15:05:56.518359 24359 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:05:56 dist-test-slave-7153 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435156, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:56.521984 24359 init.cc:377] Logged in from keytab as kudu/127.9.123.66@KRBTEST.COM (short username kudu)
I20260926 15:05:56.523824 24359 webserver.cc:460] Webserver started at http://127.9.123.66:42239/ using document root <none> and password file <none>
I20260926 15:05:56.524581 24359 fs_manager.cc:359] Metadata directory not provided
I20260926 15:05:56.524712 24359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:05:56.531051 24359 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260926 15:05:56.534564 24374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:56.536369 24359 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 15:05:56.536590 24359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "cd61584d310b455ba36e80f6178ec37b"
format_stamp: "Formatted at 2026-09-26 15:05:53 on dist-test-slave-7153"
server_key: "8c56bbf1e9883c429769438d8aedb809"
server_key_iv: "3446ffbc19b1b57693716e0a95e74c82"
server_key_version: "encryptionkey@0"
I20260926 15:05:56.537902 24359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:05:56.559468 24359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:05:56.564507 24359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:05:56.564785 24359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:05:56.565953 24359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:05:56.567471 24359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:05:56.567572 24359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:56.567684 24359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:05:56.567751 24359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:56.602661 24359 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:45037
I20260926 15:05:56.602726 24486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:45037 every 8 connection(s)
I20260926 15:05:56.604514 24359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:05:56.613574 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 24359
I20260926 15:05:56.614315 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:33221
--local_ip_for_outbound_sockets=127.9.123.67
--tserver_master_addrs=127.9.123.126:39699
--webserver_port=41789
--webserver_interface=127.9.123.67
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:44657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 15:05:56 dist-test-slave-7153 krb5kdc[23582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435156, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:05:56.627303 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:56.607309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:46735 (local address 127.9.123.126:39699)
0926 15:05:56.607514 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:56.607520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:56.608787 (+ 1267us) server_negotiation.cc:407] Connection header received
0926 15:05:56.610760 (+ 1973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:56.610767 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:56.610895 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:56.611002 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:56.612647 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:56.613810 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:56.616269 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:56.616722 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:56.616833 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:56.621287 (+ 4454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:56.621293 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:56.621296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:56.621337 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:56.623877 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:56.624732 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:56.624737 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:56.624740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:56.624882 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:56.625428 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:56.625434 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:56.625437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:56.625744 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:56.625892 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:56.626673 (+ 781us) server_negotiation.cc:299] Negotiation successful
0926 15:05:56.626935 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 15:05:56.628471 24489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:56.607774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.66:46735)
0926 15:05:56.608600 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:56.608645 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:56.610465 (+ 1820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:56.611263 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:56.611272 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:56.611816 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:05:56.612450 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:56.612470 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:56.614025 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:56.614035 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:56.616051 (+ 2016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:56.616070 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:56.616978 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:56.616986 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:56.617319 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:56.618130 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:05:56.618171 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:05:56.620939 (+ 2768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:05:56.624067 (+ 3128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:56.624078 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:56.624096 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:56.624584 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:56.625001 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:56.625007 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:56.625011 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:56.625303 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:56.625882 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:05:56.625891 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:05:56.626274 (+ 383us) client_negotiation.cc:769] Sending connection context
0926 15:05:56.626542 (+ 268us) client_negotiation.cc:240] Negotiation successful
0926 15:05:56.627154 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":334,"threads_started":1}
I20260926 15:05:56.630064 24487 heartbeater.cc:347] Connected to a master server at 127.9.123.126:39699
I20260926 15:05:56.630445 24487 heartbeater.cc:464] Registering TS with master...
I20260926 15:05:56.631421 24487 heartbeater.cc:511] Master 127.9.123.126:39699 requested a full tablet report, sending...
I20260926 15:05:56.633066 24170 ts_manager.cc:194] Registered new tserver with Master: cd61584d310b455ba36e80f6178ec37b (127.9.123.66:45037)
I20260926 15:05:56.633929 24170 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.66@KRBTEST.COM'} at 127.9.123.66:46735
W20260926 15:05:56.954988 24490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:05:56.955284 24490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:05:56.955420 24490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:05:56.963479 24490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:05:56.963629 24490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:05:56.963780 24490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:05:56.974182 24490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:44657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:33221
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=41789
--tserver_master_addrs=127.9.123.126:39699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:05:56.975520 24490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:05:56.976958 24490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:05:56.990449 24495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:56.990520 24496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:56.990576 24498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:56.991477 24490 server_base.cc:1034] running on GCE node
I20260926 15:05:56.992131 24490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:05:56.993211 24490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:05:56.994467 24490 hybrid_clock.cc:648] HybridClock initialized: now 1790435156994393 us; error 62 us; skew 500 ppm
I20260926 15:05:56.994827 24490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:05:56 dist-test-slave-7153 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435156, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:56.998576 24490 init.cc:377] Logged in from keytab as kudu/127.9.123.67@KRBTEST.COM (short username kudu)
I20260926 15:05:57.000437 24490 webserver.cc:460] Webserver started at http://127.9.123.67:41789/ using document root <none> and password file <none>
I20260926 15:05:57.001299 24490 fs_manager.cc:359] Metadata directory not provided
I20260926 15:05:57.001456 24490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:05:57.007589 24490 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 15:05:57.011245 24505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:57.012945 24490 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260926 15:05:57.013221 24490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "a94aa4e153f14ee89870daa80e6b3f30"
format_stamp: "Formatted at 2026-09-26 15:05:54 on dist-test-slave-7153"
server_key: "5fa580af0e81ce462387b18dfc5f2317"
server_key_iv: "2fd45b928397cf224eb46a8602562593"
server_key_version: "encryptionkey@0"
I20260926 15:05:57.014300 24490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:05:57.043087 24490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:05:57.047581 24490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:05:57.047870 24490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:05:57.048967 24490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:05:57.050450 24490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:05:57.050557 24490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:57.050659 24490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:05:57.050724 24490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:57.082321 24490 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:33221
I20260926 15:05:57.082320 24617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:33221 every 8 connection(s)
I20260926 15:05:57.083865 24490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:05:57.088966 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 24490
Sep 26 15:05:57 dist-test-slave-7153 krb5kdc[23582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435156, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:05:57.108826 24621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.089910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57914 (local address 127.9.123.126:39699)
0926 15:05:57.090447 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:57.090452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:57.090777 (+ 325us) server_negotiation.cc:407] Connection header received
0926 15:05:57.090934 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:57.090938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:57.090992 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:57.091074 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:57.092879 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.094015 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.095686 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.096427 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.096557 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:57.099506 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:57.099529 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:57.099535 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:57.099584 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:57.104357 (+ 4773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:57.106286 (+ 1929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:57.106292 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:57.106299 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:57.106463 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:57.106938 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:57.106943 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:57.106946 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:57.107333 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:57.107613 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:57.108197 (+ 584us) server_negotiation.cc:299] Negotiation successful
0926 15:05:57.108430 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":92,"server-negotiator.queue_time_us":424,"thread_start_us":329,"threads_started":1}
I20260926 15:05:57.109571 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.086386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:35207 (local address 127.9.123.126:39699)
0926 15:05:57.086580 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:57.086586 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:57.087756 (+ 1170us) server_negotiation.cc:407] Connection header received
0926 15:05:57.089779 (+ 2023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:57.089786 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:57.089878 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:57.089993 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:57.091795 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.092876 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.094950 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.095565 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.095758 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:57.100022 (+ 4264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:57.100028 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:57.100033 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:57.100092 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:57.104340 (+ 4248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:57.105895 (+ 1555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:57.105901 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:57.105903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:57.106076 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:57.106773 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:57.106778 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:57.106780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:57.107321 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:57.107520 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:57.108815 (+ 1295us) server_negotiation.cc:299] Negotiation successful
0926 15:05:57.109047 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":79,"server-negotiator.queue_time_us":62,"spinlock_wait_cycles":8576}
I20260926 15:05:57.111020 24620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.086881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.67:35207)
0926 15:05:57.087584 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:57.087633 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:57.089420 (+ 1787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:57.090349 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:57.090364 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:57.091002 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:05:57.091622 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:57.091642 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.093049 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.093054 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:57.094795 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:57.094810 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.095797 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.095805 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:57.096118 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:57.096960 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:05:57.096985 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:05:57.099742 (+ 2757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:05:57.104589 (+ 4847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:57.104599 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:57.104617 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:57.105651 (+ 1034us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:57.106335 (+ 684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:05:57.106340 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:05:57.106343 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:05:57.106609 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:05:57.107517 (+ 908us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:05:57.107527 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:05:57.108271 (+ 744us) client_negotiation.cc:769] Sending connection context
0926 15:05:57.108638 (+ 367us) client_negotiation.cc:240] Negotiation successful
0926 15:05:57.109461 (+ 823us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":276,"threads_started":1}
I20260926 15:05:57.112524 24618 heartbeater.cc:347] Connected to a master server at 127.9.123.126:39699
I20260926 15:05:57.112846 24618 heartbeater.cc:464] Registering TS with master...
I20260926 15:05:57.113646 24618 heartbeater.cc:511] Master 127.9.123.126:39699 requested a full tablet report, sending...
I20260926 15:05:57.115135 24170 ts_manager.cc:194] Registered new tserver with Master: a94aa4e153f14ee89870daa80e6b3f30 (127.9.123.67:33221)
I20260926 15:05:57.116122 24170 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.67@KRBTEST.COM'} at 127.9.123.67:35207
I20260926 15:05:57.125020 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 15:05:57.132788 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.126055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57926 (local address 127.9.123.126:39699)
0926 15:05:57.126291 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:57.126298 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:57.126715 (+ 417us) server_negotiation.cc:407] Connection header received
0926 15:05:57.126944 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:57.126948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:57.127147 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:57.127288 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:05:57.127761 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.128545 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.130393 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.130831 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.130954 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:57.131593 (+ 639us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:05:57.132147 (+ 554us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:05:57.132290 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:57.132414 (+ 124us) server_negotiation.cc:299] Negotiation successful
0926 15:05:57.132510 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260926 15:05:57.133963 24170 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:57926:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:05:57.141546 24170 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:05:57.162035 24355 heartbeater.cc:503] Master 127.9.123.126:39699 was elected leader, sending a full tablet report...
I20260926 15:05:57.167748 24631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.157941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:37823 (local address 127.0.0.1:60586)
0926 15:05:57.158551 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:57.158573 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:57.158698 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:57.159686 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:57.159692 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:57.159719 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:05:57.160066 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:57.160079 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.162263 (+ 2184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.162269 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:57.164552 (+ 2283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:57.164571 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.166633 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.166639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:57.166859 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:57.166929 (+ 70us) client_negotiation.cc:769] Sending connection context
0926 15:05:57.167071 (+ 142us) client_negotiation.cc:240] Negotiation successful
0926 15:05:57.167293 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":1,"thread_start_us":400,"threads_started":1}
I20260926 15:05:57.168581 24628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.156130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:33221 (local address 127.0.0.1:35338)
0926 15:05:57.157036 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:57.157093 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:57.157314 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:57.158491 (+ 1177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:57.158499 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:57.158547 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:05:57.159045 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:57.159060 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.161136 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.161142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:57.165600 (+ 4458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:57.165614 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.167569 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.167579 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:57.167776 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:57.167807 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 15:05:57.167884 (+ 77us) client_negotiation.cc:240] Negotiation successful
0926 15:05:57.168016 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"mutex_wait_us":1942,"thread_start_us":463,"threads_started":1}
I20260926 15:05:57.168924 24629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.156250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35338 (local address 127.9.123.67:33221)
0926 15:05:57.157777 (+ 1527us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:57.157786 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:57.157821 (+ 35us) server_negotiation.cc:407] Connection header received
0926 15:05:57.157924 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:57.157933 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:57.158241 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:57.158460 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:05:57.159264 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.160937 (+ 1673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.165792 (+ 4855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.167256 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.167385 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:57.168328 (+ 943us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:57.168409 (+ 81us) server_negotiation.cc:299] Negotiation successful
0926 15:05:57.168595 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1350,"thread_start_us":1184,"threads_started":1}
I20260926 15:05:57.169198 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.157612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60586 (local address 127.9.123.65:37823)
0926 15:05:57.158138 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:57.158146 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:57.158829 (+ 683us) server_negotiation.cc:407] Connection header received
0926 15:05:57.158939 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:57.158947 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:57.159246 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:57.159442 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:05:57.160278 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.162049 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.164761 (+ 2712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.166362 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.166507 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:57.166764 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:57.167132 (+ 368us) server_negotiation.cc:299] Negotiation successful
0926 15:05:57.167405 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":528,"threads_started":1}
I20260926 15:05:57.172771 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.158792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:45037 (local address 127.0.0.1:47000)
0926 15:05:57.165155 (+ 6363us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:57.165176 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:57.165318 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:57.165990 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:57.165995 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:57.166021 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:05:57.166388 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:57.166398 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.168650 (+ 2252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.168655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:57.170579 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:57.170594 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.171903 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.171909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:57.172077 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:57.172102 (+ 25us) client_negotiation.cc:769] Sending connection context
0926 15:05:57.172186 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 15:05:57.172322 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6185,"thread_start_us":6037,"threads_started":1}
I20260926 15:05:57.172816 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.159284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47000 (local address 127.9.123.66:45037)
0926 15:05:57.164643 (+ 5359us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:57.164653 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:57.165280 (+ 627us) server_negotiation.cc:407] Connection header received
0926 15:05:57.165480 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:57.165490 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:57.165801 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:57.165992 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:05:57.166553 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.168492 (+ 1939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.170740 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.171684 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.171815 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:57.172022 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:57.172261 (+ 239us) server_negotiation.cc:299] Negotiation successful
0926 15:05:57.172427 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5204,"thread_start_us":5057,"threads_started":1}
I20260926 15:05:57.173344 24553 tablet_service.cc:1467] Processing CreateTablet for tablet 9b1e825d2a9b406b832d3fb57c68157e (DEFAULT_TABLE table=test-table [id=60a0b4d47d2b49d8b33127da8dc477d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:05:57.173342 24290 tablet_service.cc:1467] Processing CreateTablet for tablet 9b1e825d2a9b406b832d3fb57c68157e (DEFAULT_TABLE table=test-table [id=60a0b4d47d2b49d8b33127da8dc477d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:05:57.174989 24553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b1e825d2a9b406b832d3fb57c68157e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:05:57.175429 24290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b1e825d2a9b406b832d3fb57c68157e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:05:57.176553 24422 tablet_service.cc:1467] Processing CreateTablet for tablet 9b1e825d2a9b406b832d3fb57c68157e (DEFAULT_TABLE table=test-table [id=60a0b4d47d2b49d8b33127da8dc477d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:05:57.177929 24422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b1e825d2a9b406b832d3fb57c68157e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:05:57.191730 24634 tablet_bootstrap.cc:492] T 9b1e825d2a9b406b832d3fb57c68157e P cd61584d310b455ba36e80f6178ec37b: Bootstrap starting.
I20260926 15:05:57.195679 24634 tablet_bootstrap.cc:654] T 9b1e825d2a9b406b832d3fb57c68157e P cd61584d310b455ba36e80f6178ec37b: Neither blocks nor log segments found. Creating new log.
I20260926 15:05:57.196880 24635 tablet_bootstrap.cc:492] T 9b1e825d2a9b406b832d3fb57c68157e P b4a122d72de34473b78cd74c6086d1dc: Bootstrap starting.
I20260926 15:05:57.197921 24636 tablet_bootstrap.cc:492] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30: Bootstrap starting.
I20260926 15:05:57.198726 24634 log.cc:824] T 9b1e825d2a9b406b832d3fb57c68157e P cd61584d310b455ba36e80f6178ec37b: Log is configured to *not* fsync() on all Append() calls
I20260926 15:05:57.201774 24636 tablet_bootstrap.cc:654] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30: Neither blocks nor log segments found. Creating new log.
I20260926 15:05:57.202346 24635 tablet_bootstrap.cc:654] T 9b1e825d2a9b406b832d3fb57c68157e P b4a122d72de34473b78cd74c6086d1dc: Neither blocks nor log segments found. Creating new log.
I20260926 15:05:57.202927 24634 tablet_bootstrap.cc:492] T 9b1e825d2a9b406b832d3fb57c68157e P cd61584d310b455ba36e80f6178ec37b: No bootstrap required, opened a new log
I20260926 15:05:57.203372 24634 ts_tablet_manager.cc:1397] T 9b1e825d2a9b406b832d3fb57c68157e P cd61584d310b455ba36e80f6178ec37b: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260926 15:05:57.203521 24636 log.cc:824] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30: Log is configured to *not* fsync() on all Append() calls
I20260926 15:05:57.203472 24635 log.cc:824] T 9b1e825d2a9b406b832d3fb57c68157e P b4a122d72de34473b78cd74c6086d1dc: Log is configured to *not* fsync() on all Append() calls
I20260926 15:05:57.206934 24635 tablet_bootstrap.cc:492] T 9b1e825d2a9b406b832d3fb57c68157e P b4a122d72de34473b78cd74c6086d1dc: No bootstrap required, opened a new log
I20260926 15:05:57.207387 24635 ts_tablet_manager.cc:1397] T 9b1e825d2a9b406b832d3fb57c68157e P b4a122d72de34473b78cd74c6086d1dc: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260926 15:05:57.207933 24636 tablet_bootstrap.cc:492] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30: No bootstrap required, opened a new log
I20260926 15:05:57.208345 24636 ts_tablet_manager.cc:1397] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260926 15:05:57.215560 24635 raft_consensus.cc:348] T 9b1e825d2a9b406b832d3fb57c68157e P b4a122d72de34473b78cd74c6086d1dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94aa4e153f14ee89870daa80e6b3f30" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33221 } } peers { permanent_uuid: "b4a122d72de34473b78cd74c6086d1dc" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 37823 } } peers { permanent_uuid: "cd61584d310b455ba36e80f6178ec37b" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45037 } }
I20260926 15:05:57.215917 24634 raft_consensus.cc:348] T 9b1e825d2a9b406b832d3fb57c68157e P cd61584d310b455ba36e80f6178ec37b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94aa4e153f14ee89870daa80e6b3f30" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33221 } } peers { permanent_uuid: "b4a122d72de34473b78cd74c6086d1dc" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 37823 } } peers { permanent_uuid: "cd61584d310b455ba36e80f6178ec37b" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45037 } }
I20260926 15:05:57.216409 24634 raft_consensus.cc:374] T 9b1e825d2a9b406b832d3fb57c68157e P cd61584d310b455ba36e80f6178ec37b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:05:57.216403 24635 raft_consensus.cc:374] T 9b1e825d2a9b406b832d3fb57c68157e P b4a122d72de34473b78cd74c6086d1dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:05:57.216530 24635 raft_consensus.cc:729] T 9b1e825d2a9b406b832d3fb57c68157e P b4a122d72de34473b78cd74c6086d1dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4a122d72de34473b78cd74c6086d1dc, State: Initialized, Role: FOLLOWER
I20260926 15:05:57.216603 24634 raft_consensus.cc:729] T 9b1e825d2a9b406b832d3fb57c68157e P cd61584d310b455ba36e80f6178ec37b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd61584d310b455ba36e80f6178ec37b, State: Initialized, Role: FOLLOWER
I20260926 15:05:57.217336 24635 consensus_queue.cc:260] T 9b1e825d2a9b406b832d3fb57c68157e P b4a122d72de34473b78cd74c6086d1dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94aa4e153f14ee89870daa80e6b3f30" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33221 } } peers { permanent_uuid: "b4a122d72de34473b78cd74c6086d1dc" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 37823 } } peers { permanent_uuid: "cd61584d310b455ba36e80f6178ec37b" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45037 } }
I20260926 15:05:57.217344 24634 consensus_queue.cc:260] T 9b1e825d2a9b406b832d3fb57c68157e P cd61584d310b455ba36e80f6178ec37b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94aa4e153f14ee89870daa80e6b3f30" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33221 } } peers { permanent_uuid: "b4a122d72de34473b78cd74c6086d1dc" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 37823 } } peers { permanent_uuid: "cd61584d310b455ba36e80f6178ec37b" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45037 } }
I20260926 15:05:57.218804 24487 heartbeater.cc:503] Master 127.9.123.126:39699 was elected leader, sending a full tablet report...
I20260926 15:05:57.219492 24634 ts_tablet_manager.cc:1428] T 9b1e825d2a9b406b832d3fb57c68157e P cd61584d310b455ba36e80f6178ec37b: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
I20260926 15:05:57.220171 24636 raft_consensus.cc:348] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94aa4e153f14ee89870daa80e6b3f30" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33221 } } peers { permanent_uuid: "b4a122d72de34473b78cd74c6086d1dc" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 37823 } } peers { permanent_uuid: "cd61584d310b455ba36e80f6178ec37b" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45037 } }
I20260926 15:05:57.221407 24636 raft_consensus.cc:374] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:05:57.221544 24636 raft_consensus.cc:729] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a94aa4e153f14ee89870daa80e6b3f30, State: Initialized, Role: FOLLOWER
I20260926 15:05:57.222110 24635 ts_tablet_manager.cc:1428] T 9b1e825d2a9b406b832d3fb57c68157e P b4a122d72de34473b78cd74c6086d1dc: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260926 15:05:57.222270 24636 consensus_queue.cc:260] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94aa4e153f14ee89870daa80e6b3f30" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33221 } } peers { permanent_uuid: "b4a122d72de34473b78cd74c6086d1dc" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 37823 } } peers { permanent_uuid: "cd61584d310b455ba36e80f6178ec37b" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45037 } }
I20260926 15:05:57.225410 24618 heartbeater.cc:503] Master 127.9.123.126:39699 was elected leader, sending a full tablet report...
I20260926 15:05:57.226227 24636 ts_tablet_manager.cc:1428] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30: Time spent starting tablet: real 0.018s user 0.012s sys 0.004s
I20260926 15:05:57.304167 24642 raft_consensus.cc:482] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:05:57.304487 24642 raft_consensus.cc:504] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94aa4e153f14ee89870daa80e6b3f30" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33221 } } peers { permanent_uuid: "b4a122d72de34473b78cd74c6086d1dc" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 37823 } } peers { permanent_uuid: "cd61584d310b455ba36e80f6178ec37b" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45037 } }
I20260926 15:05:57.306327 24642 leader_election.cc:288] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4a122d72de34473b78cd74c6086d1dc (127.9.123.65:37823), cd61584d310b455ba36e80f6178ec37b (127.9.123.66:45037)
I20260926 15:05:57.313952 24620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.306820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:37823 (local address 127.9.123.67:60075)
0926 15:05:57.307094 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:57.307119 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:57.307267 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:57.307883 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:57.307888 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:57.307917 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:05:57.308126 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:57.308140 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.309445 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.309449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:57.311424 (+ 1975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:57.311441 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.313142 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.313153 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:57.313355 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:57.313395 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 15:05:57.313500 (+ 105us) client_negotiation.cc:240] Negotiation successful
0926 15:05:57.313655 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260926 15:05:57.313975 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.306965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:60075 (local address 127.9.123.65:37823)
0926 15:05:57.307175 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:57.307181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:57.307203 (+ 22us) server_negotiation.cc:407] Connection header received
0926 15:05:57.307465 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:57.307472 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:57.307549 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:57.307660 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:05:57.308316 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.309303 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.311600 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.312708 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.312895 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:57.312976 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:57.313576 (+ 600us) server_negotiation.cc:299] Negotiation successful
0926 15:05:57.313726 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 15:05:57.314198 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.307537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:32929 (local address 127.9.123.66:45037)
0926 15:05:57.307795 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:57.307800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:57.308191 (+ 391us) server_negotiation.cc:407] Connection header received
0926 15:05:57.308286 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:57.308291 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:57.308370 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:57.308476 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:05:57.309008 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.309840 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.312045 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.313240 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.313392 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:57.313467 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:57.313810 (+ 343us) server_negotiation.cc:299] Negotiation successful
0926 15:05:57.313945 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260926 15:05:57.314877 24643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.307296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:45037 (local address 127.9.123.67:32929)
0926 15:05:57.308031 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:57.308067 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0926 15:05:57.308188 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:05:57.308523 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:05:57.308526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:05:57.308550 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:05:57.308852 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:57.308860 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.309983 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.309988 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:57.311875 (+ 1887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:05:57.311886 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.313445 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.313452 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:05:57.313625 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:57.313658 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 15:05:57.314308 (+ 650us) client_negotiation.cc:240] Negotiation successful
0926 15:05:57.314458 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":386,"threads_started":1}
I20260926 15:05:57.315513 24310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b1e825d2a9b406b832d3fb57c68157e" candidate_uuid: "a94aa4e153f14ee89870daa80e6b3f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4a122d72de34473b78cd74c6086d1dc" is_pre_election: true
I20260926 15:05:57.316071 24310 raft_consensus.cc:2454] T 9b1e825d2a9b406b832d3fb57c68157e P b4a122d72de34473b78cd74c6086d1dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a94aa4e153f14ee89870daa80e6b3f30 in term 0.
I20260926 15:05:57.316010 24442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b1e825d2a9b406b832d3fb57c68157e" candidate_uuid: "a94aa4e153f14ee89870daa80e6b3f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd61584d310b455ba36e80f6178ec37b" is_pre_election: true
I20260926 15:05:57.316493 24442 raft_consensus.cc:2454] T 9b1e825d2a9b406b832d3fb57c68157e P cd61584d310b455ba36e80f6178ec37b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a94aa4e153f14ee89870daa80e6b3f30 in term 0.
I20260926 15:05:57.316893 24507 leader_election.cc:302] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a94aa4e153f14ee89870daa80e6b3f30, b4a122d72de34473b78cd74c6086d1dc; no voters:
I20260926 15:05:57.317493 24642 raft_consensus.cc:2781] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:05:57.317638 24642 raft_consensus.cc:482] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:05:57.317742 24642 raft_consensus.cc:3037] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:05:57.320350 24642 raft_consensus.cc:504] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94aa4e153f14ee89870daa80e6b3f30" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33221 } } peers { permanent_uuid: "b4a122d72de34473b78cd74c6086d1dc" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 37823 } } peers { permanent_uuid: "cd61584d310b455ba36e80f6178ec37b" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45037 } }
I20260926 15:05:57.321080 24642 leader_election.cc:288] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30 [CANDIDATE]: Term 1 election: Requested vote from peers b4a122d72de34473b78cd74c6086d1dc (127.9.123.65:37823), cd61584d310b455ba36e80f6178ec37b (127.9.123.66:45037)
I20260926 15:05:57.321796 24310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b1e825d2a9b406b832d3fb57c68157e" candidate_uuid: "a94aa4e153f14ee89870daa80e6b3f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4a122d72de34473b78cd74c6086d1dc"
I20260926 15:05:57.321925 24442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b1e825d2a9b406b832d3fb57c68157e" candidate_uuid: "a94aa4e153f14ee89870daa80e6b3f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd61584d310b455ba36e80f6178ec37b"
I20260926 15:05:57.322028 24310 raft_consensus.cc:3037] T 9b1e825d2a9b406b832d3fb57c68157e P b4a122d72de34473b78cd74c6086d1dc [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:05:57.322201 24442 raft_consensus.cc:3037] T 9b1e825d2a9b406b832d3fb57c68157e P cd61584d310b455ba36e80f6178ec37b [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:05:57.324422 24310 raft_consensus.cc:2454] T 9b1e825d2a9b406b832d3fb57c68157e P b4a122d72de34473b78cd74c6086d1dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a94aa4e153f14ee89870daa80e6b3f30 in term 1.
I20260926 15:05:57.324479 24442 raft_consensus.cc:2454] T 9b1e825d2a9b406b832d3fb57c68157e P cd61584d310b455ba36e80f6178ec37b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a94aa4e153f14ee89870daa80e6b3f30 in term 1.
I20260926 15:05:57.324955 24507 leader_election.cc:302] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a94aa4e153f14ee89870daa80e6b3f30, b4a122d72de34473b78cd74c6086d1dc; no voters:
I20260926 15:05:57.325366 24642 raft_consensus.cc:2781] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:05:57.325917 24642 raft_consensus.cc:686] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30 [term 1 LEADER]: Becoming Leader. State: Replica: a94aa4e153f14ee89870daa80e6b3f30, State: Running, Role: LEADER
I20260926 15:05:57.326547 24642 consensus_queue.cc:237] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94aa4e153f14ee89870daa80e6b3f30" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33221 } } peers { permanent_uuid: "b4a122d72de34473b78cd74c6086d1dc" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 37823 } } peers { permanent_uuid: "cd61584d310b455ba36e80f6178ec37b" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45037 } }
I20260926 15:05:57.332526 24169 catalog_manager.cc:5495] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30 reported cstate change: term changed from 0 to 1, leader changed from <none> to a94aa4e153f14ee89870daa80e6b3f30 (127.9.123.67). New cstate: current_term: 1 leader_uuid: "a94aa4e153f14ee89870daa80e6b3f30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a94aa4e153f14ee89870daa80e6b3f30" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4a122d72de34473b78cd74c6086d1dc" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 37823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd61584d310b455ba36e80f6178ec37b" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45037 } health_report { overall_health: UNKNOWN } } }
W20260926 15:05:57.335536 24619 tablet.cc:2369] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 15:05:57.356761 24488 tablet.cc:2369] T 9b1e825d2a9b406b832d3fb57c68157e P cd61584d310b455ba36e80f6178ec37b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:05:57.366667 24629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.360170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35352 (local address 127.9.123.67:33221)
0926 15:05:57.360408 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:57.360414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:57.360483 (+ 69us) server_negotiation.cc:407] Connection header received
0926 15:05:57.360731 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:57.360734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:57.360829 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:57.360988 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:05:57.361757 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.362503 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.364105 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.364529 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.364662 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:57.365372 (+ 710us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:05:57.366003 (+ 631us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:05:57.366160 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:57.366293 (+ 133us) server_negotiation.cc:299] Negotiation successful
0926 15:05:57.366394 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260926 15:05:57.374823 24356 tablet.cc:2369] T 9b1e825d2a9b406b832d3fb57c68157e P b4a122d72de34473b78cd74c6086d1dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:05:57.376382 24310 raft_consensus.cc:1261] T 9b1e825d2a9b406b832d3fb57c68157e P b4a122d72de34473b78cd74c6086d1dc [term 1 FOLLOWER]: Refusing update from remote peer a94aa4e153f14ee89870daa80e6b3f30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:05:57.376601 24442 raft_consensus.cc:1261] T 9b1e825d2a9b406b832d3fb57c68157e P cd61584d310b455ba36e80f6178ec37b [term 1 FOLLOWER]: Refusing update from remote peer a94aa4e153f14ee89870daa80e6b3f30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:05:57.377542 24642 consensus_queue.cc:1035] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4a122d72de34473b78cd74c6086d1dc" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 37823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:05:57.377933 24644 consensus_queue.cc:1035] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd61584d310b455ba36e80f6178ec37b" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:05:57.411760 24169 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:57926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:05:57.412065 24169 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:57926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:05:57.416249 24169 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b: Sending DeleteTablet for 3 replicas of tablet 9b1e825d2a9b406b832d3fb57c68157e
I20260926 15:05:57.417652 24553 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b1e825d2a9b406b832d3fb57c68157e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:05:57 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:35338
I20260926 15:05:57.418053 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 24207
I20260926 15:05:57.418141 24422 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b1e825d2a9b406b832d3fb57c68157e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:05:57 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:47000
I20260926 15:05:57.418511 24290 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b1e825d2a9b406b832d3fb57c68157e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:05:57 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:60586
I20260926 15:05:57.418922 24656 tablet_replica.cc:330] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30: stopping tablet replica
I20260926 15:05:57.419147 24657 tablet_replica.cc:330] T 9b1e825d2a9b406b832d3fb57c68157e P cd61584d310b455ba36e80f6178ec37b: stopping tablet replica
I20260926 15:05:57.419575 24657 raft_consensus.cc:2229] T 9b1e825d2a9b406b832d3fb57c68157e P cd61584d310b455ba36e80f6178ec37b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:05:57.419577 24656 raft_consensus.cc:2229] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30 [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:05:57.420156 24657 raft_consensus.cc:2258] T 9b1e825d2a9b406b832d3fb57c68157e P cd61584d310b455ba36e80f6178ec37b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:05:57.420969 24656 raft_consensus.cc:2258] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:05:57.423384 24657 ts_tablet_manager.cc:1905] T 9b1e825d2a9b406b832d3fb57c68157e P cd61584d310b455ba36e80f6178ec37b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:05:57.428650 24656 ts_tablet_manager.cc:1905] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:05:57.429442 24658 tablet_replica.cc:330] T 9b1e825d2a9b406b832d3fb57c68157e P b4a122d72de34473b78cd74c6086d1dc: stopping tablet replica
I20260926 15:05:57.430109 24658 raft_consensus.cc:2229] T 9b1e825d2a9b406b832d3fb57c68157e P b4a122d72de34473b78cd74c6086d1dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:05:57.430805 24658 raft_consensus.cc:2258] T 9b1e825d2a9b406b832d3fb57c68157e P b4a122d72de34473b78cd74c6086d1dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:05:57.433739 24658 ts_tablet_manager.cc:1905] T 9b1e825d2a9b406b832d3fb57c68157e P b4a122d72de34473b78cd74c6086d1dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:05:57.434316 24657 ts_tablet_manager.cc:1918] T 9b1e825d2a9b406b832d3fb57c68157e P cd61584d310b455ba36e80f6178ec37b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:05:57.434607 24657 log.cc:1196] T 9b1e825d2a9b406b832d3fb57c68157e P cd61584d310b455ba36e80f6178ec37b: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/9b1e825d2a9b406b832d3fb57c68157e
I20260926 15:05:57.435289 24657 ts_tablet_manager.cc:1939] T 9b1e825d2a9b406b832d3fb57c68157e P cd61584d310b455ba36e80f6178ec37b: Deleting consensus metadata
I20260926 15:05:57.437862 24155 catalog_manager.cc:4841] TS cd61584d310b455ba36e80f6178ec37b (127.9.123.66:45037): tablet 9b1e825d2a9b406b832d3fb57c68157e (table test-table [id=60a0b4d47d2b49d8b33127da8dc477d1]) successfully deleted
I20260926 15:05:57.440925 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.420058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60596 (local address 127.9.123.65:37823)
0926 15:05:57.420286 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:57.420292 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:57.420318 (+ 26us) server_negotiation.cc:407] Connection header received
0926 15:05:57.420381 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:57.420389 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:57.420467 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:57.420570 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:57.422947 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.424458 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.427074 (+ 2616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.427767 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.427923 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:57.431835 (+ 3912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:57.431849 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:57.431866 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:57.431921 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:57.436012 (+ 4091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:57.437312 (+ 1300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:57.437324 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:57.437327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:57.437529 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:57.438087 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:57.438093 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:57.438096 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:57.438591 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:57.438926 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:57.440237 (+ 1311us) server_negotiation.cc:299] Negotiation successful
0926 15:05:57.440503 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260926 15:05:57.441682 24350 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:05:57.442994 24656 ts_tablet_manager.cc:1918] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:05:57.443207 24656 log.cc:1196] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/9b1e825d2a9b406b832d3fb57c68157e
I20260926 15:05:57.443830 24656 ts_tablet_manager.cc:1939] T 9b1e825d2a9b406b832d3fb57c68157e P a94aa4e153f14ee89870daa80e6b3f30: Deleting consensus metadata
I20260926 15:05:57.446070 24156 catalog_manager.cc:4841] TS a94aa4e153f14ee89870daa80e6b3f30 (127.9.123.67:33221): tablet 9b1e825d2a9b406b832d3fb57c68157e (table test-table [id=60a0b4d47d2b49d8b33127da8dc477d1]) successfully deleted
I20260926 15:05:57.590122 24658 ts_tablet_manager.cc:1918] T 9b1e825d2a9b406b832d3fb57c68157e P b4a122d72de34473b78cd74c6086d1dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:05:57.590322 24658 log.cc:1196] T 9b1e825d2a9b406b832d3fb57c68157e P b4a122d72de34473b78cd74c6086d1dc: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/wal/wals/9b1e825d2a9b406b832d3fb57c68157e
I20260926 15:05:57.590762 24658 ts_tablet_manager.cc:1939] T 9b1e825d2a9b406b832d3fb57c68157e P b4a122d72de34473b78cd74c6086d1dc: Deleting consensus metadata
I20260926 15:05:57.591095 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 24207
W20260926 15:05:57.612380 24155 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.123.65:37823 (error 108)
I20260926 15:05:57.618651 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 24359
I20260926 15:05:57.619949 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.619439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:37823 (local address 127.0.0.1:60604)
0926 15:05:57.619644 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:57.619777 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:37823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 15:05:57.620478 24155 catalog_manager.cc:4584] TS b4a122d72de34473b78cd74c6086d1dc (127.9.123.65:37823): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9b1e825d2a9b406b832d3fb57c68157e: Network error: Client connection negotiation failed: client connection to 127.9.123.65:37823: connect: Connection refused (error 111)
I20260926 15:05:57.637540 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.620216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47010 (local address 127.9.123.66:45037)
0926 15:05:57.620404 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:57.620410 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:57.620431 (+ 21us) server_negotiation.cc:407] Connection header received
0926 15:05:57.620495 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:57.620501 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:57.620605 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:57.620716 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:57.622684 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.623823 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.625904 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.626578 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.626703 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:57.630369 (+ 3666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:57.630384 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:57.630399 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:57.630437 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:57.633734 (+ 3297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:57.634877 (+ 1143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:57.634887 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:57.634889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:57.635085 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:57.635611 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:57.635616 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:57.635618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:57.636238 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:57.636502 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:57.636944 (+ 442us) server_negotiation.cc:299] Negotiation successful
0926 15:05:57.637213 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 15:05:57.638265 24482 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:05:57.674510 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.673884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:37823 (local address 127.0.0.1:60616)
0926 15:05:57.674193 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:57.674329 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:37823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260926 15:05:57.675737 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.675316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:37823 (local address 127.0.0.1:60624)
0926 15:05:57.675487 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:57.675579 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:37823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260926 15:05:57.731694 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 24359
I20260926 15:05:57.736361 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.735850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:37823 (local address 127.0.0.1:60632)
0926 15:05:57.736077 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:57.736190 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:37823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260926 15:05:57.737423 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.737058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:37823 (local address 127.0.0.1:60644)
0926 15:05:57.737233 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:57.737309 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:37823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260926 15:05:57.752188 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 24490
I20260926 15:05:57.769410 24629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.753079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35354 (local address 127.9.123.67:33221)
0926 15:05:57.753283 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 15:05:57.753288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:05:57.753406 (+ 118us) server_negotiation.cc:407] Connection header received
0926 15:05:57.753569 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:05:57.753574 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:05:57.753653 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:05:57.753752 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:05:57.755907 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.756739 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.758911 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:05:57.759400 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:05:57.759522 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:05:57.763049 (+ 3527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:05:57.763062 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:05:57.763077 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:05:57.763123 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:05:57.766201 (+ 3078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:57.767255 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:57.767263 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:57.767266 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:57.767441 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:05:57.767855 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:05:57.767858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:05:57.767860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:05:57.768256 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:05:57.768516 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0926 15:05:57.768856 (+ 340us) server_negotiation.cc:299] Negotiation successful
0926 15:05:57.769032 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260926 15:05:57.770251 24613 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:05:57.819072 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.818416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:37823 (local address 127.0.0.1:60658)
0926 15:05:57.818758 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:57.818874 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:37823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201}
I20260926 15:05:57.820499 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:57.819959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:37823 (local address 127.0.0.1:60672)
0926 15:05:57.820267 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:57.820356 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:37823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260926 15:05:57.903743 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 24490
I20260926 15:05:57.921070 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 24137
I20260926 15:05:57.922574 24200 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:05:58.038941 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 24137
I20260926 15:05:58.039052 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:58.038536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:37823 (local address 127.0.0.1:60676)
0926 15:05:58.038766 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:58.038874 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:37823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260926 15:05:58.057845 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:39699
--webserver_interface=127.9.123.126
--webserver_port=34565
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:44657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:39699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:05:58.410810 24664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:05:58.411126 24664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:05:58.411224 24664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:05:58.419104 24664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:05:58.419252 24664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:05:58.419301 24664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:05:58.419384 24664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:05:58.419441 24664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:05:58.430512 24664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:44657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:39699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:39699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=34565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:05:58.432031 24664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:05:58.433452 24664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:05:58.446233 24669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:58.446297 24672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:58.446197 24670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:58.448105 24664 server_base.cc:1034] running on GCE node
I20260926 15:05:58.449065 24664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:05:58.450852 24664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:05:58.452087 24664 hybrid_clock.cc:648] HybridClock initialized: now 1790435158452012 us; error 75 us; skew 500 ppm
I20260926 15:05:58.452491 24664 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:05:58 dist-test-slave-7153 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435158, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:58.455971 24664 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:05:58.457682 24664 webserver.cc:460] Webserver started at http://127.9.123.126:34565/ using document root <none> and password file <none>
I20260926 15:05:58.458420 24664 fs_manager.cc:359] Metadata directory not provided
I20260926 15:05:58.458542 24664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:05:58.464856 24664 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260926 15:05:58.468446 24679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:58.469956 24664 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:05:58.470160 24664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "f1b5473193be4bba9ed73a4ef5eae05b"
format_stamp: "Formatted at 2026-09-26 15:05:52 on dist-test-slave-7153"
server_key: "ef60863b8c1e4db4b0c7973ace27d81e"
server_key_iv: "23aaed99722434801033d129cbbaee7d"
server_key_version: "encryptionkey@0"
I20260926 15:05:58.471235 24664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:05:58.484426 24664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:05:58.489018 24664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:05:58.489329 24664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:05:58.508172 24664 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:39699
I20260926 15:05:58.508181 24730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:39699 every 8 connection(s)
I20260926 15:05:58.509845 24664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:05:58.511696 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 24664
I20260926 15:05:58.512682 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:37823
--local_ip_for_outbound_sockets=127.9.123.65
--tserver_master_addrs=127.9.123.126:39699
--webserver_port=37973
--webserver_interface=127.9.123.65
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:44657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:05:58.519771 24731 sys_catalog.cc:263] Verifying existing consensus state
I20260926 15:05:58.523969 24731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b: Bootstrap starting.
I20260926 15:05:58.545691 24731 log.cc:824] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b: Log is configured to *not* fsync() on all Append() calls
I20260926 15:05:58.562497 24731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 15:05:58.563117 24731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b: Bootstrap complete.
I20260926 15:05:58.572924 24731 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b5473193be4bba9ed73a4ef5eae05b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39699 } }
I20260926 15:05:58.574009 24731 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1b5473193be4bba9ed73a4ef5eae05b, State: Initialized, Role: FOLLOWER
I20260926 15:05:58.574630 24731 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b5473193be4bba9ed73a4ef5eae05b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39699 } }
I20260926 15:05:58.574862 24731 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:05:58.575006 24731 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:05:58.575194 24731 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 2 FOLLOWER]: Advancing to term 3
I20260926 15:05:58.579435 24731 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b5473193be4bba9ed73a4ef5eae05b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39699 } }
I20260926 15:05:58.580017 24731 leader_election.cc:302] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1b5473193be4bba9ed73a4ef5eae05b; no voters:
I20260926 15:05:58.580730 24731 leader_election.cc:288] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 15:05:58.580881 24735 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 3 FOLLOWER]: Leader election won for term 3
I20260926 15:05:58.583341 24731 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:05:58.585611 24735 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [term 3 LEADER]: Becoming Leader. State: Replica: f1b5473193be4bba9ed73a4ef5eae05b, State: Running, Role: LEADER
I20260926 15:05:58.586500 24735 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b5473193be4bba9ed73a4ef5eae05b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39699 } }
I20260926 15:05:58.589375 24737 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1b5473193be4bba9ed73a4ef5eae05b. Latest consensus state: current_term: 3 leader_uuid: "f1b5473193be4bba9ed73a4ef5eae05b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b5473193be4bba9ed73a4ef5eae05b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39699 } } }
I20260926 15:05:58.589780 24737 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [sys.catalog]: This master's current role is: LEADER
I20260926 15:05:58.590394 24736 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "f1b5473193be4bba9ed73a4ef5eae05b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b5473193be4bba9ed73a4ef5eae05b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39699 } } }
I20260926 15:05:58.590708 24736 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1b5473193be4bba9ed73a4ef5eae05b [sys.catalog]: This master's current role is: LEADER
I20260926 15:05:58.595705 24747 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:05:58.606410 24747 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:05:58.608414 24747 catalog_manager.cc:1259] Loaded cluster ID: 07a465c2cd634c1e8cc51a2cb3135d52
I20260926 15:05:58.608676 24747 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260926 15:05:58.611301 24747 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f2b6409c9ad google::LogMessage::Fail() at ??:0
@ 0x7f2b6409f88a google::LogMessage::SendToLog() at ??:0
@ 0x7f2b6409c4de google::LogMessage::Flush() at ??:0
@ 0x7f2b6409da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f2b79acf512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f2b79acc976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f2b79b54610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f2b79b54439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f2b66ce8345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f2b66ceb91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f2b66ceb799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f2b66cc6eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2b69fa06db start_thread at ??:0
@ 0x7f2b62f3661f clone at ??:0
W20260926 15:05:58.902714 24733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:05:58.903045 24733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:05:58.903182 24733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:05:58.910928 24733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:05:58.911085 24733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:05:58.911248 24733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:05:58.921573 24733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:44657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:37823
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=37973
--tserver_master_addrs=127.9.123.126:39699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:05:58.922942 24733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:05:58.924329 24733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:05:58.938694 24772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:58.938767 24775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:58.938725 24773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:58.939797 24733 server_base.cc:1034] running on GCE node
I20260926 15:05:58.940544 24733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:05:58.941684 24733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:05:58.942948 24733 hybrid_clock.cc:648] HybridClock initialized: now 1790435158942877 us; error 47 us; skew 500 ppm
I20260926 15:05:58.943356 24733 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:05:58 dist-test-slave-7153 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435158, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:58.946940 24733 init.cc:377] Logged in from keytab as kudu/127.9.123.65@KRBTEST.COM (short username kudu)
I20260926 15:05:58.948582 24733 webserver.cc:460] Webserver started at http://127.9.123.65:37973/ using document root <none> and password file <none>
I20260926 15:05:58.949399 24733 fs_manager.cc:359] Metadata directory not provided
I20260926 15:05:58.949514 24733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:05:58.955334 24733 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260926 15:05:58.958397 24782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:58.960142 24733 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 15:05:58.960321 24733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "b4a122d72de34473b78cd74c6086d1dc"
format_stamp: "Formatted at 2026-09-26 15:05:53 on dist-test-slave-7153"
server_key: "44fa0d2ab79b8b5a5f35ff3e7ba3e1fc"
server_key_iv: "03c41bc14fe4f82f0827b1b60b7cfd43"
server_key_version: "encryptionkey@0"
I20260926 15:05:58.961426 24733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:05:58.997354 24733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:05:59.002315 24733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:05:59.002600 24733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:05:59.003657 24733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:05:59.005118 24733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:05:59.005214 24733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:59.005335 24733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:05:59.005402 24733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:59.037547 24733 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:37823
I20260926 15:05:59.037618 24894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:37823 every 8 connection(s)
I20260926 15:05:59.039070 24733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:05:59.040194 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 24733
I20260926 15:05:59.040880 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:45037
--local_ip_for_outbound_sockets=127.9.123.66
--tserver_master_addrs=127.9.123.126:39699
--webserver_port=42239
--webserver_interface=127.9.123.66
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:44657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:05:59.044471 24897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:59.042326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.65:55579)
0926 15:05:59.043175 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:59.043475 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":339,"threads_started":1}
W20260926 15:05:59.045200 24895 heartbeater.cc:644] Failed to heartbeat to 127.9.123.126:39699 (0 consecutive failures): Network error: Failed to ping master at 127.9.123.126:39699: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
I20260926 15:05:59.046113 24897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:59.045769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.65:42745)
0926 15:05:59.045927 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:59.046001 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260926 15:05:59.046983 24897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:59.046616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.65:42799)
0926 15:05:59.046799 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:59.046872 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260926 15:05:59.047217 24895 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 15:05:59.397801 24898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:05:59.398115 24898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:05:59.398216 24898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:05:59.405910 24898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:05:59.406033 24898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:05:59.406184 24898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:05:59.416411 24898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:44657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:45037
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=42239
--tserver_master_addrs=127.9.123.126:39699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:05:59.417853 24898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:05:59.419307 24898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:05:59.434324 24906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:59.434492 24904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:59.434868 24903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:59.435037 24898 server_base.cc:1034] running on GCE node
I20260926 15:05:59.435719 24898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:05:59.436801 24898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:05:59.438019 24898 hybrid_clock.cc:648] HybridClock initialized: now 1790435159437952 us; error 64 us; skew 500 ppm
I20260926 15:05:59.438376 24898 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:05:59 dist-test-slave-7153 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435159, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:59.441733 24898 init.cc:377] Logged in from keytab as kudu/127.9.123.66@KRBTEST.COM (short username kudu)
I20260926 15:05:59.443439 24898 webserver.cc:460] Webserver started at http://127.9.123.66:42239/ using document root <none> and password file <none>
I20260926 15:05:59.444195 24898 fs_manager.cc:359] Metadata directory not provided
I20260926 15:05:59.444326 24898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:05:59.450284 24898 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 15:05:59.453678 24913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:59.455386 24898 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260926 15:05:59.455597 24898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "cd61584d310b455ba36e80f6178ec37b"
format_stamp: "Formatted at 2026-09-26 15:05:53 on dist-test-slave-7153"
server_key: "8c56bbf1e9883c429769438d8aedb809"
server_key_iv: "3446ffbc19b1b57693716e0a95e74c82"
server_key_version: "encryptionkey@0"
I20260926 15:05:59.456698 24898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:05:59.474408 24898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:05:59.478672 24898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:05:59.478943 24898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:05:59.479967 24898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:05:59.481365 24898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:05:59.481469 24898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:59.481580 24898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:05:59.481652 24898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:59.513320 24898 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:45037
I20260926 15:05:59.513408 25025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:45037 every 8 connection(s)
I20260926 15:05:59.515472 24898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:05:59.521275 25028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:59.518938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.66:44307)
0926 15:05:59.519795 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:59.520043 (+ 248us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":339,"threads_started":1}
W20260926 15:05:59.522171 25026 heartbeater.cc:644] Failed to heartbeat to 127.9.123.126:39699 (0 consecutive failures): Network error: Failed to ping master at 127.9.123.126:39699: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
I20260926 15:05:59.523180 25028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:59.522738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.66:56499)
0926 15:05:59.522965 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:59.523049 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260926 15:05:59.524263 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 24898
I20260926 15:05:59.525035 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:33221
--local_ip_for_outbound_sockets=127.9.123.67
--tserver_master_addrs=127.9.123.126:39699
--webserver_port=41789
--webserver_interface=127.9.123.67
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:44657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:05:59.525744 25028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:05:59.525195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.66:41651)
0926 15:05:59.525456 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0926 15:05:59.525559 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
W20260926 15:05:59.526077 25026 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 15:05:59.880362 25029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:05:59.880681 25029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:05:59.880784 25029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:05:59.888990 25029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:05:59.889209 25029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:05:59.889366 25029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:05:59.899709 25029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:44657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:33221
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=41789
--tserver_master_addrs=127.9.123.126:39699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:05:59.901141 25029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:05:59.902833 25029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:05:59.918493 25034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:59.918437 25037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:05:59.918467 25035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:05:59.919582 25029 server_base.cc:1034] running on GCE node
I20260926 15:05:59.920553 25029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:05:59.921860 25029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:05:59.923115 25029 hybrid_clock.cc:648] HybridClock initialized: now 1790435159923047 us; error 60 us; skew 500 ppm
I20260926 15:05:59.923513 25029 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:05:59 dist-test-slave-7153 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435159, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:05:59.927129 25029 init.cc:377] Logged in from keytab as kudu/127.9.123.67@KRBTEST.COM (short username kudu)
I20260926 15:05:59.928957 25029 webserver.cc:460] Webserver started at http://127.9.123.67:41789/ using document root <none> and password file <none>
I20260926 15:05:59.929755 25029 fs_manager.cc:359] Metadata directory not provided
I20260926 15:05:59.929884 25029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:05:59.936810 25029 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 15:05:59.940302 25044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:59.942070 25029 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 15:05:59.942273 25029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "a94aa4e153f14ee89870daa80e6b3f30"
format_stamp: "Formatted at 2026-09-26 15:05:54 on dist-test-slave-7153"
server_key: "5fa580af0e81ce462387b18dfc5f2317"
server_key_iv: "2fd45b928397cf224eb46a8602562593"
server_key_version: "encryptionkey@0"
I20260926 15:05:59.943655 25029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:05:59.967571 25029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:05:59.972359 25029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:05:59.972648 25029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:05:59.973932 25029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:05:59.975355 25029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:05:59.975458 25029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:05:59.975564 25029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:05:59.975629 25029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:06:00.008868 25029 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:33221
I20260926 15:06:00.008960 25156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:33221 every 8 connection(s)
I20260926 15:06:00.010494 25029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:06:00.014355 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 25029
I20260926 15:06:00.016120 25159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:00.013892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.67:48015)
0926 15:06:00.014830 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:00.015147 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":362,"threads_started":1}
W20260926 15:06:00.016896 25157 heartbeater.cc:644] Failed to heartbeat to 127.9.123.126:39699 (0 consecutive failures): Network error: Failed to ping master at 127.9.123.126:39699: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
I20260926 15:06:00.017777 25159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:00.017431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.67:60535)
0926 15:06:00.017588 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:00.017659 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260926 15:06:00.018640 25159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:00.018309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.67:34711)
0926 15:06:00.018449 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:00.018523 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 15:06:00.018872 25157 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 15:06:00.048909 25161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:00.047947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.65:33189)
0926 15:06:00.048523 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:00.048632 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":319,"threads_started":1}
I20260926 15:06:00.527931 25162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:00.526841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.66:39175)
0926 15:06:00.527526 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:00.527676 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":404,"threads_started":1}
I20260926 15:06:01.020426 25163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:01.019523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.67:41985)
0926 15:06:01.020022 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:01.020123 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":250,"threads_started":1}
I20260926 15:06:01.050724 25164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:01.049913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.65:60111)
0926 15:06:01.050438 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:01.050532 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":272,"threads_started":1}
I20260926 15:06:01.529798 25165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:01.529009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.66:59875)
0926 15:06:01.529534 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:01.529625 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":276,"threads_started":1}
I20260926 15:06:02.022482 25166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:02.021652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.67:50347)
0926 15:06:02.022195 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:02.022290 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":242,"threads_started":1}
I20260926 15:06:02.052909 25167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:02.051930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.65:57831)
0926 15:06:02.052482 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:02.052591 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":296,"threads_started":1}
I20260926 15:06:02.531555 25168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:02.530791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.66:59231)
0926 15:06:02.531273 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:02.531369 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":234,"threads_started":1}
I20260926 15:06:03.024291 25169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:03.023520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.67:39747)
0926 15:06:03.023993 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:03.024087 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":248,"threads_started":1}
I20260926 15:06:03.054807 25170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:03.053977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.65:49067)
0926 15:06:03.054528 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:03.054619 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":313,"threads_started":1}
I20260926 15:06:03.533644 25171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:03.532623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.66:48765)
0926 15:06:03.533306 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:03.533409 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":308,"threads_started":1}
I20260926 15:06:04.026706 25172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:04.025656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.67:34035)
0926 15:06:04.026269 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:04.026446 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":377,"threads_started":1}
I20260926 15:06:04.056653 25173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:04.055827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.65:56991)
0926 15:06:04.056355 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:04.056465 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":261,"threads_started":1}
I20260926 15:06:04.535516 25174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:04.534738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.66:40365)
0926 15:06:04.535235 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:04.535334 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":237,"threads_started":1}
I20260926 15:06:05.028702 25175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:05.027774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.67:50441)
0926 15:06:05.028402 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:05.028525 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":274,"threads_started":1}
I20260926 15:06:05.058629 25176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:05.057811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.65:43815)
0926 15:06:05.058347 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:05.058457 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":261,"threads_started":1}
I20260926 15:06:05.537621 25177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:05.536565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.66:33731)
0926 15:06:05.537089 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:05.537412 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":255,"threads_started":1}
I20260926 15:06:06.030453 25178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:06.029670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.67:40161)
0926 15:06:06.030165 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:06.030278 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":260,"threads_started":1}
I20260926 15:06:06.060423 25179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:06.059623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.65:49075)
0926 15:06:06.060102 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:06.060187 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":198,"threads_started":1}
I20260926 15:06:06.539539 25180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:06.538674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.66:60825)
0926 15:06:06.539210 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:06.539304 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":236,"threads_started":1}
I20260926 15:06:07.032445 25181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:07.031573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.67:46367)
0926 15:06:07.032143 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:07.032227 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":266,"threads_started":1}
I20260926 15:06:07.062240 25182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:07.061526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.65:48455)
0926 15:06:07.061977 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:07.062072 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":221,"threads_started":1}
I20260926 15:06:07.541564 25183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:07.540576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.66:45257)
0926 15:06:07.541216 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:07.541354 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":313,"threads_started":1}
I20260926 15:06:08.034329 25184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:08.033512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.67:34059)
0926 15:06:08.034016 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:08.034111 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":251,"threads_started":1}
I20260926 15:06:08.063894 25185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:08.063100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.65:58349)
0926 15:06:08.063598 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:08.063717 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":235,"threads_started":1}
I20260926 15:06:08.543443 25186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:08.542598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.66:38605)
0926 15:06:08.543168 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:08.543260 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":287,"threads_started":1}
I20260926 15:06:09.036155 25187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:09.035285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.67:38115)
0926 15:06:09.035888 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:09.035980 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":252,"threads_started":1}
I20260926 15:06:09.065763 25188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:09.064922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.65:58389)
0926 15:06:09.065496 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:09.065592 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":339,"threads_started":1}
I20260926 15:06:09.545210 25189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:09.544460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.66:56915)
0926 15:06:09.544904 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:09.545004 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":235,"threads_started":1}
I20260926 15:06:10.038064 25190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:10.037239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.67:38963)
0926 15:06:10.037776 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:10.037884 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":291,"threads_started":1}
I20260926 15:06:10.067449 25191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:10.066721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.65:52075)
0926 15:06:10.067169 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:10.067265 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":217,"threads_started":1}
I20260926 15:06:10.546957 25192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:10.546178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.66:35347)
0926 15:06:10.546662 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:10.546753 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":254,"threads_started":1}
I20260926 15:06:11.039822 25193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:11.039094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.67:44339)
0926 15:06:11.039543 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:11.039646 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":214,"threads_started":1}
I20260926 15:06:11.069051 25194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:11.068339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.65:38299)
0926 15:06:11.068802 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:11.068890 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":248,"threads_started":1}
I20260926 15:06:11.548650 25195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:11.547928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.66:56313)
0926 15:06:11.548401 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:11.548491 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":237,"threads_started":1}
I20260926 15:06:12.041640 25196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:12.040843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.67:51873)
0926 15:06:12.041345 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:12.041485 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":287,"threads_started":1}
I20260926 15:06:12.070848 25197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:12.070049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.65:38475)
0926 15:06:12.070557 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:12.070665 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":248,"threads_started":1}
I20260926 15:06:12.550462 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:12.549679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.66:48015)
0926 15:06:12.550152 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:12.550254 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":221,"threads_started":1}
I20260926 15:06:13.043450 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:13.042608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.67:33263)
0926 15:06:13.043102 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:13.043234 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":258,"threads_started":1}
I20260926 15:06:13.072584 25200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:13.071786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.65:50603)
0926 15:06:13.072259 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:13.072350 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":235,"threads_started":1}
I20260926 15:06:13.552253 25201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:13.551438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.66:42989)
0926 15:06:13.551965 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:13.552066 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":259,"threads_started":1}
I20260926 15:06:14.045308 25202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:14.044456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.67:49963)
0926 15:06:14.044966 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:14.045058 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":269,"threads_started":1}
I20260926 15:06:14.075095 25203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:14.074204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.65:50811)
0926 15:06:14.074780 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:14.074884 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":325,"threads_started":1}
I20260926 15:06:14.554286 25204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:14.553517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.66:35815)
0926 15:06:14.553998 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:14.554100 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":227,"threads_started":1}
I20260926 15:06:15.021585 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 24733
I20260926 15:06:15.047155 25205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:15.022742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47998 (local address 127.9.123.65:37823)
0926 15:06:15.023450 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:15.023469 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:15.023524 (+ 55us) server_negotiation.cc:407] Connection header received
0926 15:06:15.025271 (+ 1747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:15.025282 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:15.025739 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:15.026059 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:06:15.027538 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:15.028847 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:15.030270 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:15.031032 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:15.031172 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:15.040069 (+ 8897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:06:15.040100 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:06:15.040126 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:06:15.040189 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:06:15.043774 (+ 3585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:15.044573 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:15.044582 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:15.044589 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:15.044786 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:15.045135 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:15.045140 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:15.045143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:15.045746 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:06:15.046081 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:15.046421 (+ 340us) server_negotiation.cc:299] Negotiation successful
0926 15:06:15.046812 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":345,"threads_started":1}
I20260926 15:06:15.047473 25206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:15.046567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39699 (local address 127.9.123.67:54283)
0926 15:06:15.047158 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:15.047268 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:39699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":366,"threads_started":1}
I20260926 15:06:15.048763 24890 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:06:15.131942 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 24733
I20260926 15:06:15.145829 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 24898
I20260926 15:06:15.163638 25209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:15.146716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47216 (local address 127.9.123.66:45037)
0926 15:06:15.147226 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:15.147241 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:15.147278 (+ 37us) server_negotiation.cc:407] Connection header received
0926 15:06:15.148904 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:15.148914 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:15.149400 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:15.149755 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:06:15.151197 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:15.152392 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:15.153726 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:15.154436 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:15.154610 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:15.157277 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:06:15.157286 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:06:15.157304 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:06:15.157348 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:06:15.160118 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:15.160923 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:15.160931 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:15.160938 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:15.161175 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:15.161571 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:15.161576 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:15.161581 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:15.162152 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:06:15.162441 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:15.162894 (+ 453us) server_negotiation.cc:299] Negotiation successful
0926 15:06:15.163336 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":227,"threads_started":1}
I20260926 15:06:15.165068 25021 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:06:15.248090 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 24898
I20260926 15:06:15.262239 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 25029
I20260926 15:06:15.280547 25211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:15.263365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60244 (local address 127.9.123.67:33221)
0926 15:06:15.264051 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:15.264066 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:15.264109 (+ 43us) server_negotiation.cc:407] Connection header received
0926 15:06:15.265858 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:15.265870 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:15.266329 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:15.266635 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:06:15.267955 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:15.269375 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:15.270635 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:15.271299 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:15.271450 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:15.274270 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:06:15.274281 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:06:15.274300 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:06:15.274351 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:06:15.277354 (+ 3003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:15.278058 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:15.278065 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:15.278072 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:15.278232 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:15.278582 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:15.278585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:15.278587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:15.279112 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:06:15.279335 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:15.279847 (+ 512us) server_negotiation.cc:299] Negotiation successful
0926 15:06:15.280207 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":362,"threads_started":1}
I20260926 15:06:15.282068 25152 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:06:15.365474 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 25029
2026-09-26T15:06:15Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24499 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:06:15 dist-test-slave-7153 krb5kdc[25215](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:06:15 dist-test-slave-7153 krb5kdc[25215](info): set up 2 sockets
Sep 26 15:06:15 dist-test-slave-7153 krb5kdc[25215](info): commencing operation
krb5kdc: starting...
W20260926 15:06:16.385543 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.983s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:06:16 dist-test-slave-7153 krb5kdc[25215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:06:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:06:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:06:16.621179 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:45713
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:33955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:45713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:06:16.957583 25228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:06:16.957918 25228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:06:16.958040 25228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:06:16.965634 25228 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 15:06:16.965764 25228 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 15:06:16.965819 25228 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 15:06:16.965886 25228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:06:16.965963 25228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:06:16.966066 25228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:06:16.966125 25228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:06:16.966198 25228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:06:16.976754 25228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:33955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:45713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:45713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:06:16.978168 25228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:06:16.979583 25228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:06:16.991950 25233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:06:16.991973 25234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:06:16.992241 25228 server_base.cc:1034] running on GCE node
W20260926 15:06:16.991922 25236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:06:16.993489 25228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:06:16.995007 25228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:06:16.996249 25228 hybrid_clock.cc:648] HybridClock initialized: now 1790435176996183 us; error 44 us; skew 500 ppm
I20260926 15:06:16.996622 25228 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:06:16 dist-test-slave-7153 krb5kdc[25215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435176, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:06:16.999850 25228 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:06:17.001590 25228 webserver.cc:460] Webserver started at http://127.9.123.126:41349/ using document root <none> and password file <none>
I20260926 15:06:17.002338 25228 fs_manager.cc:359] Metadata directory not provided
I20260926 15:06:17.002461 25228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:06:17.002763 25228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:06:17.005946 25228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "51190f416ba240538a331f103c888f15"
format_stamp: "Formatted at 2026-09-26 15:06:16 on dist-test-slave-7153"
server_key: "ee27df8aec007c1b2827e5e464497254"
server_key_iv: "11f4ff4135b2044dcf5af14d87bbe19b"
server_key_version: "encryptionkey@0"
I20260926 15:06:17.006749 25228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "51190f416ba240538a331f103c888f15"
format_stamp: "Formatted at 2026-09-26 15:06:16 on dist-test-slave-7153"
server_key: "ee27df8aec007c1b2827e5e464497254"
server_key_iv: "11f4ff4135b2044dcf5af14d87bbe19b"
server_key_version: "encryptionkey@0"
I20260926 15:06:17.012264 25228 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 15:06:17.016373 25243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:06:17.018002 25228 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:06:17.018167 25228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "51190f416ba240538a331f103c888f15"
format_stamp: "Formatted at 2026-09-26 15:06:16 on dist-test-slave-7153"
server_key: "ee27df8aec007c1b2827e5e464497254"
server_key_iv: "11f4ff4135b2044dcf5af14d87bbe19b"
server_key_version: "encryptionkey@0"
I20260926 15:06:17.018368 25228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:06:17.032081 25228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:06:17.036458 25228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:06:17.036706 25228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:06:17.054335 25228 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:45713
I20260926 15:06:17.054354 25294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:45713 every 8 connection(s)
I20260926 15:06:17.055825 25228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:06:17.059888 25295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:06:17.063876 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 25228
I20260926 15:06:17.064086 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:06:17.064728 9709 external_mini_cluster.cc:1428] Setting key c40df5a0c62a5631020dcfce4e63587e
I20260926 15:06:17.073153 25295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15: Bootstrap starting.
I20260926 15:06:17.076462 25295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15: Neither blocks nor log segments found. Creating new log.
I20260926 15:06:17.077612 25295 log.cc:824] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15: Log is configured to *not* fsync() on all Append() calls
Sep 26 15:06:17 dist-test-slave-7153 krb5kdc[25215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:06:17.081089 25295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15: No bootstrap required, opened a new log
I20260926 15:06:17.089584 25298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:17.066846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49174 (local address 127.9.123.126:45713)
0926 15:06:17.067694 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:17.067708 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:17.067753 (+ 45us) server_negotiation.cc:407] Connection header received
0926 15:06:17.069532 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:17.069566 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:17.070040 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:17.070456 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:06:17.071969 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:17.073402 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:17.074769 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:17.075483 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:17.075696 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:17.080109 (+ 4413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:06:17.080122 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:06:17.080145 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:06:17.080201 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:06:17.084283 (+ 4082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:17.085204 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:17.085215 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:17.085226 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:17.085468 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:17.085901 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:17.085906 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:17.085910 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:17.086736 (+ 826us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:06:17.087084 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:17.087513 (+ 429us) server_negotiation.cc:299] Negotiation successful
0926 15:06:17.088048 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":365,"threads_started":1}
I20260926 15:06:17.091908 25295 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51190f416ba240538a331f103c888f15" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45713 } }
I20260926 15:06:17.092293 25295 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:06:17.092403 25295 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51190f416ba240538a331f103c888f15, State: Initialized, Role: FOLLOWER
I20260926 15:06:17.093093 25295 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51190f416ba240538a331f103c888f15" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45713 } }
I20260926 15:06:17.093377 25295 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:06:17.093525 25295 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:06:17.093672 25295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:06:17.096175 25295 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51190f416ba240538a331f103c888f15" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45713 } }
I20260926 15:06:17.096719 25295 leader_election.cc:302] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51190f416ba240538a331f103c888f15; no voters:
I20260926 15:06:17.097355 25295 leader_election.cc:288] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:06:17.097498 25300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:06:17.098141 25300 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 1 LEADER]: Becoming Leader. State: Replica: 51190f416ba240538a331f103c888f15, State: Running, Role: LEADER
I20260926 15:06:17.098783 25300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51190f416ba240538a331f103c888f15" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45713 } }
I20260926 15:06:17.099386 25295 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:06:17.101563 25302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51190f416ba240538a331f103c888f15. Latest consensus state: current_term: 1 leader_uuid: "51190f416ba240538a331f103c888f15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51190f416ba240538a331f103c888f15" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45713 } } }
I20260926 15:06:17.102010 25302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [sys.catalog]: This master's current role is: LEADER
I20260926 15:06:17.102118 25301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51190f416ba240538a331f103c888f15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51190f416ba240538a331f103c888f15" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45713 } } }
I20260926 15:06:17.102413 25301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [sys.catalog]: This master's current role is: LEADER
I20260926 15:06:17.103050 25308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:06:17.107836 25308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:06:17.117376 25308 catalog_manager.cc:1347] Generated new cluster ID: 4c182619a59440af9eb564a4b411c7e0
I20260926 15:06:17.117527 25308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:06:17.133795 25308 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:06:17.135188 25308 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:06:17.141283 25308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15: Generated new TSK 0
I20260926 15:06:17.142246 25308 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.123.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
I20260926 15:06:17.211092 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:45713
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:33955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:06:17.552304 25321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:06:17.552620 25321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:06:17.552742 25321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:06:17.560295 25321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:06:17.560431 25321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:06:17.560593 25321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:06:17.570815 25321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:33955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:45713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:06:17.572144 25321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:06:17.573601 25321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:06:17.586997 25326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:06:17.587097 25327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:06:17.587097 25329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:06:17.588450 25321 server_base.cc:1034] running on GCE node
I20260926 15:06:17.589067 25321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:06:17.590095 25321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:06:17.591303 25321 hybrid_clock.cc:648] HybridClock initialized: now 1790435177591251 us; error 42 us; skew 500 ppm
I20260926 15:06:17.591661 25321 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:06:17 dist-test-slave-7153 krb5kdc[25215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435177, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:06:17.594743 25321 init.cc:377] Logged in from keytab as kudu/127.9.123.65@KRBTEST.COM (short username kudu)
I20260926 15:06:17.596196 25321 webserver.cc:460] Webserver started at http://127.9.123.65:35783/ using document root <none> and password file <none>
I20260926 15:06:17.596861 25321 fs_manager.cc:359] Metadata directory not provided
I20260926 15:06:17.596968 25321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:06:17.597307 25321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:06:17.600243 25321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "3bd7598679d84b62bbb2a0082717b051"
format_stamp: "Formatted at 2026-09-26 15:06:17 on dist-test-slave-7153"
server_key: "a7836fa31b6310506ee5d532b937c04b"
server_key_iv: "afd5550bca1e9f1400b925636321072f"
server_key_version: "encryptionkey@0"
I20260926 15:06:17.601019 25321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "3bd7598679d84b62bbb2a0082717b051"
format_stamp: "Formatted at 2026-09-26 15:06:17 on dist-test-slave-7153"
server_key: "a7836fa31b6310506ee5d532b937c04b"
server_key_iv: "afd5550bca1e9f1400b925636321072f"
server_key_version: "encryptionkey@0"
I20260926 15:06:17.606388 25321 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 15:06:17.610037 25336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:06:17.611557 25321 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:06:17.611723 25321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "3bd7598679d84b62bbb2a0082717b051"
format_stamp: "Formatted at 2026-09-26 15:06:17 on dist-test-slave-7153"
server_key: "a7836fa31b6310506ee5d532b937c04b"
server_key_iv: "afd5550bca1e9f1400b925636321072f"
server_key_version: "encryptionkey@0"
I20260926 15:06:17.611919 25321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:06:17.630430 25321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:06:17.635282 25321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:06:17.635582 25321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:06:17.636613 25321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:06:17.638067 25321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:06:17.638139 25321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:06:17.638243 25321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:06:17.638295 25321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:06:17.670084 25321 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:45717
I20260926 15:06:17.670145 25448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:45717 every 8 connection(s)
I20260926 15:06:17.671382 25321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:06:17.678809 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 25321
I20260926 15:06:17.679049 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:06:17.679747 9709 external_mini_cluster.cc:1428] Setting key 8da9458931493a7a44cfff18931dea61
Sep 26 15:06:17 dist-test-slave-7153 krb5kdc[25215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435177, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:06:17.692737 25451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:17.674013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:58599 (local address 127.9.123.126:45713)
0926 15:06:17.674441 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:17.674445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:17.675492 (+ 1047us) server_negotiation.cc:407] Connection header received
0926 15:06:17.677381 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:17.677386 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:17.677457 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:17.677552 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:06:17.679360 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:17.680157 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:17.681842 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:17.682250 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:17.682344 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:17.686559 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:06:17.686565 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:06:17.686569 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:06:17.686615 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:06:17.689255 (+ 2640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:17.690162 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:17.690167 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:17.690170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:17.690317 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:17.690870 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:17.690874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:17.690876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:17.691174 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:06:17.691291 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:17.692252 (+ 961us) server_negotiation.cc:299] Negotiation successful
0926 15:06:17.692454 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":189,"threads_started":1}
I20260926 15:06:17.694069 25452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:17.674627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.65:58599)
0926 15:06:17.675327 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:17.675375 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 15:06:17.677064 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:06:17.677762 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:06:17.677770 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:06:17.678289 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:06:17.679143 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:17.679170 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:17.680323 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:17.680328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:17.681691 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:17.681707 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:17.682479 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:17.682485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:17.682773 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:17.683483 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:06:17.683528 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:06:17.686269 (+ 2741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:06:17.689428 (+ 3159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:06:17.689440 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:06:17.689461 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:06:17.689980 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:06:17.690438 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:06:17.690443 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:06:17.690447 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:06:17.690732 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:06:17.691288 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:06:17.691296 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:06:17.691776 (+ 480us) client_negotiation.cc:769] Sending connection context
0926 15:06:17.692105 (+ 329us) client_negotiation.cc:240] Negotiation successful
0926 15:06:17.692676 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":262,"threads_started":1}
I20260926 15:06:17.695824 25449 heartbeater.cc:347] Connected to a master server at 127.9.123.126:45713
I20260926 15:06:17.696141 25449 heartbeater.cc:464] Registering TS with master...
I20260926 15:06:17.696930 25449 heartbeater.cc:511] Master 127.9.123.126:45713 requested a full tablet report, sending...
I20260926 15:06:17.699560 25260 ts_manager.cc:194] Registered new tserver with Master: 3bd7598679d84b62bbb2a0082717b051 (127.9.123.65:45717)
I20260926 15:06:17.701941 25260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.65@KRBTEST.COM'} at 127.9.123.65:58599
WARNING: no policy specified for kudu/127.9.123.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
I20260926 15:06:17.736569 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:45713
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:33955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:06:18.069761 25455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:06:18.070075 25455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:06:18.070204 25455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:06:18.078066 25455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:06:18.078233 25455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:06:18.078384 25455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:06:18.088562 25455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:33955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:45713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:06:18.089950 25455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:06:18.091311 25455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:06:18.105244 25460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:06:18.105168 25463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:06:18.105170 25461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:06:18.106383 25455 server_base.cc:1034] running on GCE node
I20260926 15:06:18.107059 25455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:06:18.108054 25455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:06:18.109261 25455 hybrid_clock.cc:648] HybridClock initialized: now 1790435178109200 us; error 47 us; skew 500 ppm
I20260926 15:06:18.109704 25455 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:06:18 dist-test-slave-7153 krb5kdc[25215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435178, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:06:18.112926 25455 init.cc:377] Logged in from keytab as kudu/127.9.123.66@KRBTEST.COM (short username kudu)
I20260926 15:06:18.114513 25455 webserver.cc:460] Webserver started at http://127.9.123.66:39617/ using document root <none> and password file <none>
I20260926 15:06:18.115223 25455 fs_manager.cc:359] Metadata directory not provided
I20260926 15:06:18.115346 25455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:06:18.115633 25455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:06:18.118705 25455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "e823b55004f64499978fb94e96d2f89f"
format_stamp: "Formatted at 2026-09-26 15:06:18 on dist-test-slave-7153"
server_key: "4df59d307b65f0a0a439a903c9d9fac2"
server_key_iv: "790190b5d108337d13f192be265264a2"
server_key_version: "encryptionkey@0"
I20260926 15:06:18.119467 25455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "e823b55004f64499978fb94e96d2f89f"
format_stamp: "Formatted at 2026-09-26 15:06:18 on dist-test-slave-7153"
server_key: "4df59d307b65f0a0a439a903c9d9fac2"
server_key_iv: "790190b5d108337d13f192be265264a2"
server_key_version: "encryptionkey@0"
I20260926 15:06:18.125233 25455 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260926 15:06:18.129154 25470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:06:18.130815 25455 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:06:18.130982 25455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "e823b55004f64499978fb94e96d2f89f"
format_stamp: "Formatted at 2026-09-26 15:06:18 on dist-test-slave-7153"
server_key: "4df59d307b65f0a0a439a903c9d9fac2"
server_key_iv: "790190b5d108337d13f192be265264a2"
server_key_version: "encryptionkey@0"
I20260926 15:06:18.131170 25455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:06:18.158607 25455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:06:18.163293 25455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:06:18.163555 25455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:06:18.164511 25455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:06:18.165908 25455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:06:18.165987 25455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:06:18.166082 25455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:06:18.166124 25455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:06:18.196756 25455 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:42499
I20260926 15:06:18.196842 25582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:42499 every 8 connection(s)
I20260926 15:06:18.198104 25455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:06:18.199365 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 25455
I20260926 15:06:18.199540 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:06:18.199973 9709 external_mini_cluster.cc:1428] Setting key 67dfb71a514fda8a8e138329e3f3d0e8
Sep 26 15:06:18 dist-test-slave-7153 krb5kdc[25215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435178, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:06:18.219511 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:18.201046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:48461 (local address 127.9.123.126:45713)
0926 15:06:18.201501 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:18.201506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:18.202756 (+ 1250us) server_negotiation.cc:407] Connection header received
0926 15:06:18.204816 (+ 2060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:18.204822 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:18.204911 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:18.205011 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:06:18.206767 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.207504 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.209405 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.209870 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.209993 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:18.213635 (+ 3642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:06:18.213640 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:06:18.213644 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:06:18.213681 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:06:18.216295 (+ 2614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:18.217077 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:18.217082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:18.217085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:18.217266 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:18.217713 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:18.217718 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:18.217722 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:18.218001 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:06:18.218154 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:18.218886 (+ 732us) server_negotiation.cc:299] Negotiation successful
0926 15:06:18.219139 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":221,"threads_started":1}
I20260926 15:06:18.220551 25586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:18.201783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.66:48461)
0926 15:06:18.202577 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:18.202629 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 15:06:18.204494 (+ 1865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:06:18.205349 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:06:18.205362 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:06:18.205949 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:06:18.206565 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:18.206594 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.207680 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.207684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:18.209215 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:18.209237 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.210089 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.210093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:18.210328 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:18.210988 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:06:18.211011 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:06:18.213372 (+ 2361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:06:18.216442 (+ 3070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:06:18.216452 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:06:18.216472 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:06:18.216932 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:06:18.217378 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:06:18.217383 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:06:18.217387 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:06:18.217594 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:06:18.218125 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:06:18.218133 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:06:18.218501 (+ 368us) client_negotiation.cc:769] Sending connection context
0926 15:06:18.218761 (+ 260us) client_negotiation.cc:240] Negotiation successful
0926 15:06:18.219326 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":307,"threads_started":1}
WARNING: no policy specified for kudu/127.9.123.67@KRBTEST.COM; defaulting to no policy
I20260926 15:06:18.222273 25583 heartbeater.cc:347] Connected to a master server at 127.9.123.126:45713
I20260926 15:06:18.222599 25583 heartbeater.cc:464] Registering TS with master...
I20260926 15:06:18.223446 25583 heartbeater.cc:511] Master 127.9.123.126:45713 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.67@KRBTEST.COM" created.
I20260926 15:06:18.225241 25260 ts_manager.cc:194] Registered new tserver with Master: e823b55004f64499978fb94e96d2f89f (127.9.123.66:42499)
I20260926 15:06:18.226369 25260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.66@KRBTEST.COM'} at 127.9.123.66:48461
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
I20260926 15:06:18.254323 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:45713
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:33955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:06:18.580688 25589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:06:18.581007 25589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:06:18.581172 25589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:06:18.588590 25589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:06:18.588732 25589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:06:18.588884 25589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:06:18.599094 25589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:33955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:45713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:06:18.600466 25589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:06:18.601872 25589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:06:18.615767 25594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:06:18.615731 25595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:06:18.615731 25597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:06:18.616817 25589 server_base.cc:1034] running on GCE node
I20260926 15:06:18.617568 25589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:06:18.618675 25589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:06:18.619894 25589 hybrid_clock.cc:648] HybridClock initialized: now 1790435178619822 us; error 65 us; skew 500 ppm
I20260926 15:06:18.620242 25589 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:06:18 dist-test-slave-7153 krb5kdc[25215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435178, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:06:18.623445 25589 init.cc:377] Logged in from keytab as kudu/127.9.123.67@KRBTEST.COM (short username kudu)
I20260926 15:06:18.624984 25589 webserver.cc:460] Webserver started at http://127.9.123.67:37819/ using document root <none> and password file <none>
I20260926 15:06:18.625731 25589 fs_manager.cc:359] Metadata directory not provided
I20260926 15:06:18.625856 25589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:06:18.626173 25589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:06:18.629191 25589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "2566a89ae54442eca72b9b8add797384"
format_stamp: "Formatted at 2026-09-26 15:06:18 on dist-test-slave-7153"
server_key: "0a63ffd4a54320d83bb914d3e888a06e"
server_key_iv: "243c624b9f50610ab8f3304a3ec28830"
server_key_version: "encryptionkey@0"
I20260926 15:06:18.629971 25589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "2566a89ae54442eca72b9b8add797384"
format_stamp: "Formatted at 2026-09-26 15:06:18 on dist-test-slave-7153"
server_key: "0a63ffd4a54320d83bb914d3e888a06e"
server_key_iv: "243c624b9f50610ab8f3304a3ec28830"
server_key_version: "encryptionkey@0"
I20260926 15:06:18.636330 25589 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260926 15:06:18.640340 25604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:06:18.641856 25589 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:06:18.642035 25589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "2566a89ae54442eca72b9b8add797384"
format_stamp: "Formatted at 2026-09-26 15:06:18 on dist-test-slave-7153"
server_key: "0a63ffd4a54320d83bb914d3e888a06e"
server_key_iv: "243c624b9f50610ab8f3304a3ec28830"
server_key_version: "encryptionkey@0"
I20260926 15:06:18.642241 25589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:06:18.654546 25589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:06:18.659274 25589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:06:18.659680 25589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:06:18.660730 25589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:06:18.662137 25589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:06:18.662214 25589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:06:18.662333 25589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:06:18.662387 25589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:06:18.693246 25589 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:37657
I20260926 15:06:18.693297 25716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:37657 every 8 connection(s)
I20260926 15:06:18.694530 25589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:06:18.697254 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 25589
I20260926 15:06:18.697459 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:06:18.697906 9709 external_mini_cluster.cc:1428] Setting key 2049d5fe8f690af211933ef9c2a28a44
I20260926 15:06:18.706660 25449 heartbeater.cc:503] Master 127.9.123.126:45713 was elected leader, sending a full tablet report...
Sep 26 15:06:18 dist-test-slave-7153 krb5kdc[25215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435178, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:06:18.715058 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:18.697174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:40541 (local address 127.9.123.126:45713)
0926 15:06:18.697393 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:18.697400 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:18.698636 (+ 1236us) server_negotiation.cc:407] Connection header received
0926 15:06:18.700800 (+ 2164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:18.700806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:18.700903 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:18.700985 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:06:18.702560 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.703327 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.704961 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.705422 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.705575 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:18.709173 (+ 3598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:06:18.709179 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:06:18.709182 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:06:18.709216 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:06:18.711866 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:18.712706 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:18.712711 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:18.712715 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:18.712844 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:18.713395 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:18.713401 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:18.713405 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:18.713702 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:06:18.713828 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:18.714575 (+ 747us) server_negotiation.cc:299] Negotiation successful
0926 15:06:18.714777 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 15:06:18.716045 25719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:18.697638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.67:40541)
0926 15:06:18.698466 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:18.698513 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 15:06:18.700542 (+ 2029us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:06:18.701261 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:06:18.701270 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:06:18.701786 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:06:18.702388 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:18.702406 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.703483 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.703487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:18.704811 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:18.704830 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.705660 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.705668 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:18.705910 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:18.706574 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:06:18.706599 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:06:18.708916 (+ 2317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:06:18.712002 (+ 3086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:06:18.712010 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:06:18.712027 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:06:18.712595 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:06:18.712948 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:06:18.712953 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:06:18.712956 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:06:18.713286 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:06:18.713805 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:06:18.713811 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:06:18.714170 (+ 359us) client_negotiation.cc:769] Sending connection context
0926 15:06:18.714407 (+ 237us) client_negotiation.cc:240] Negotiation successful
0926 15:06:18.714828 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":332,"threads_started":1}
I20260926 15:06:18.717454 25717 heartbeater.cc:347] Connected to a master server at 127.9.123.126:45713
I20260926 15:06:18.717770 25717 heartbeater.cc:464] Registering TS with master...
I20260926 15:06:18.718477 25717 heartbeater.cc:511] Master 127.9.123.126:45713 requested a full tablet report, sending...
I20260926 15:06:18.719980 25260 ts_manager.cc:194] Registered new tserver with Master: 2566a89ae54442eca72b9b8add797384 (127.9.123.67:37657)
I20260926 15:06:18.720769 25260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.67@KRBTEST.COM'} at 127.9.123.67:40541
I20260926 15:06:18.725873 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 15:06:18.743695 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:18.730952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49188 (local address 127.9.123.126:45713)
0926 15:06:18.731131 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:18.731135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:18.731154 (+ 19us) server_negotiation.cc:407] Connection header received
0926 15:06:18.731348 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:18.731358 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:18.731437 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:18.731561 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:06:18.732956 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.733798 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.735286 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.735773 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.735885 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:18.738412 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:06:18.738418 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:06:18.738421 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:06:18.738462 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:06:18.740906 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:18.741830 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:18.741835 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:18.741837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:18.741989 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:18.742391 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:18.742394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:18.742396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:18.742722 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:06:18.742920 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:18.743204 (+ 284us) server_negotiation.cc:299] Negotiation successful
0926 15:06:18.743384 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 15:06:18.747916 25260 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:06:18.755007 25260 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:06:18.793026 25731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:18.768058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44872 (local address 127.9.123.66:42499)
0926 15:06:18.783343 (+ 15285us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:18.783353 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:18.783386 (+ 33us) server_negotiation.cc:407] Connection header received
0926 15:06:18.783497 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:18.783506 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:18.783813 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:18.784044 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:06:18.786600 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.788130 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.790899 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.791853 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.791985 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:18.792206 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:18.792518 (+ 312us) server_negotiation.cc:299] Negotiation successful
0926 15:06:18.792693 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15103,"thread_start_us":18258,"threads_started":1}
I20260926 15:06:18.793012 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:18.767955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:42499 (local address 127.0.0.1:44872)
0926 15:06:18.782046 (+ 14091us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:18.782065 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:06:18.782172 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:06:18.784385 (+ 2213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:06:18.784395 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:06:18.784443 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:06:18.784904 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:18.784917 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.788298 (+ 3381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.788316 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:18.790729 (+ 2413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:18.790741 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.792085 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.792092 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:18.792300 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:18.792339 (+ 39us) client_negotiation.cc:769] Sending connection context
0926 15:06:18.792454 (+ 115us) client_negotiation.cc:240] Negotiation successful
0926 15:06:18.792665 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13976,"thread_start_us":13838,"threads_started":1}
I20260926 15:06:18.795285 25730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:18.767670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:45717 (local address 127.0.0.1:37480)
0926 15:06:18.781222 (+ 13552us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:18.781299 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0926 15:06:18.781540 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:06:18.787412 (+ 5872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:06:18.787416 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:06:18.787438 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:06:18.787805 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:18.787818 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.789686 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.789692 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:18.791985 (+ 2293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:18.792005 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.794380 (+ 2375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.794386 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:18.794564 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:18.794597 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 15:06:18.794685 (+ 88us) client_negotiation.cc:240] Negotiation successful
0926 15:06:18.794826 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13144,"mutex_wait_us":33,"thread_start_us":14563,"threads_started":1}
I20260926 15:06:18.795555 25732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:18.768015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37480 (local address 127.9.123.65:45717)
0926 15:06:18.785507 (+ 17492us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:18.785516 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:18.785553 (+ 37us) server_negotiation.cc:407] Connection header received
0926 15:06:18.785642 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:18.785650 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:18.785907 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:18.786109 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:06:18.788005 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.789500 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.792191 (+ 2691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.793860 (+ 1669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.793993 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:18.794228 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:18.795040 (+ 812us) server_negotiation.cc:299] Negotiation successful
0926 15:06:18.795225 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17314,"thread_start_us":17172,"threads_started":1}
I20260926 15:06:18.795893 25733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:18.782968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49798 (local address 127.9.123.67:37657)
0926 15:06:18.787552 (+ 4584us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:18.787559 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:18.787588 (+ 29us) server_negotiation.cc:407] Connection header received
0926 15:06:18.787663 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:18.787668 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:18.787847 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:18.788038 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:06:18.788650 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.790159 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.792971 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.794340 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.794459 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:18.794744 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:18.795405 (+ 661us) server_negotiation.cc:299] Negotiation successful
0926 15:06:18.795596 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4417,"thread_start_us":4268,"threads_started":1}
I20260926 15:06:18.796267 25734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:18.784066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:37657 (local address 127.0.0.1:49798)
0926 15:06:18.787156 (+ 3090us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:18.787175 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:06:18.787281 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:06:18.788037 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:06:18.788041 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:06:18.788066 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:06:18.788463 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:18.788474 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.790323 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.790328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:18.792786 (+ 2458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:18.792804 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.794873 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.794878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:18.795048 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:18.795079 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 15:06:18.795170 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 15:06:18.795302 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2967,"thread_start_us":2814,"threads_started":1}
I20260926 15:06:18.797372 25518 tablet_service.cc:1467] Processing CreateTablet for tablet 83e480b3b9134583b967ae954a90ce76 (DEFAULT_TABLE table=test-table [id=8908949602a947debeb0f6056077d7fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:06:18.798914 25518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83e480b3b9134583b967ae954a90ce76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:06:18.798985 25384 tablet_service.cc:1467] Processing CreateTablet for tablet 83e480b3b9134583b967ae954a90ce76 (DEFAULT_TABLE table=test-table [id=8908949602a947debeb0f6056077d7fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:06:18.800519 25384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83e480b3b9134583b967ae954a90ce76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:06:18.800012 25652 tablet_service.cc:1467] Processing CreateTablet for tablet 83e480b3b9134583b967ae954a90ce76 (DEFAULT_TABLE table=test-table [id=8908949602a947debeb0f6056077d7fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:06:18.801458 25652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83e480b3b9134583b967ae954a90ce76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:06:18.814450 25735 tablet_bootstrap.cc:492] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051: Bootstrap starting.
I20260926 15:06:18.815299 25736 tablet_bootstrap.cc:492] T 83e480b3b9134583b967ae954a90ce76 P e823b55004f64499978fb94e96d2f89f: Bootstrap starting.
I20260926 15:06:18.816846 25737 tablet_bootstrap.cc:492] T 83e480b3b9134583b967ae954a90ce76 P 2566a89ae54442eca72b9b8add797384: Bootstrap starting.
I20260926 15:06:18.819105 25736 tablet_bootstrap.cc:654] T 83e480b3b9134583b967ae954a90ce76 P e823b55004f64499978fb94e96d2f89f: Neither blocks nor log segments found. Creating new log.
I20260926 15:06:18.819103 25735 tablet_bootstrap.cc:654] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051: Neither blocks nor log segments found. Creating new log.
I20260926 15:06:18.819661 25737 tablet_bootstrap.cc:654] T 83e480b3b9134583b967ae954a90ce76 P 2566a89ae54442eca72b9b8add797384: Neither blocks nor log segments found. Creating new log.
I20260926 15:06:18.820427 25736 log.cc:824] T 83e480b3b9134583b967ae954a90ce76 P e823b55004f64499978fb94e96d2f89f: Log is configured to *not* fsync() on all Append() calls
I20260926 15:06:18.820427 25735 log.cc:824] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051: Log is configured to *not* fsync() on all Append() calls
I20260926 15:06:18.820603 25737 log.cc:824] T 83e480b3b9134583b967ae954a90ce76 P 2566a89ae54442eca72b9b8add797384: Log is configured to *not* fsync() on all Append() calls
I20260926 15:06:18.823836 25735 tablet_bootstrap.cc:492] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051: No bootstrap required, opened a new log
I20260926 15:06:18.823834 25736 tablet_bootstrap.cc:492] T 83e480b3b9134583b967ae954a90ce76 P e823b55004f64499978fb94e96d2f89f: No bootstrap required, opened a new log
I20260926 15:06:18.824136 25735 ts_tablet_manager.cc:1397] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.004s
I20260926 15:06:18.824136 25736 ts_tablet_manager.cc:1397] T 83e480b3b9134583b967ae954a90ce76 P e823b55004f64499978fb94e96d2f89f: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260926 15:06:18.824371 25737 tablet_bootstrap.cc:492] T 83e480b3b9134583b967ae954a90ce76 P 2566a89ae54442eca72b9b8add797384: No bootstrap required, opened a new log
I20260926 15:06:18.824609 25737 ts_tablet_manager.cc:1397] T 83e480b3b9134583b967ae954a90ce76 P 2566a89ae54442eca72b9b8add797384: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260926 15:06:18.831897 25737 raft_consensus.cc:348] T 83e480b3b9134583b967ae954a90ce76 P 2566a89ae54442eca72b9b8add797384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } } peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } }
I20260926 15:06:18.832312 25737 raft_consensus.cc:374] T 83e480b3b9134583b967ae954a90ce76 P 2566a89ae54442eca72b9b8add797384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:06:18.832405 25737 raft_consensus.cc:729] T 83e480b3b9134583b967ae954a90ce76 P 2566a89ae54442eca72b9b8add797384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2566a89ae54442eca72b9b8add797384, State: Initialized, Role: FOLLOWER
I20260926 15:06:18.833060 25737 consensus_queue.cc:260] T 83e480b3b9134583b967ae954a90ce76 P 2566a89ae54442eca72b9b8add797384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } } peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } }
I20260926 15:06:18.834887 25735 raft_consensus.cc:348] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } } peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } }
I20260926 15:06:18.834887 25736 raft_consensus.cc:348] T 83e480b3b9134583b967ae954a90ce76 P e823b55004f64499978fb94e96d2f89f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } } peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } }
I20260926 15:06:18.835402 25735 raft_consensus.cc:374] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:06:18.835402 25736 raft_consensus.cc:374] T 83e480b3b9134583b967ae954a90ce76 P e823b55004f64499978fb94e96d2f89f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:06:18.835501 25735 raft_consensus.cc:729] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd7598679d84b62bbb2a0082717b051, State: Initialized, Role: FOLLOWER
I20260926 15:06:18.835501 25736 raft_consensus.cc:729] T 83e480b3b9134583b967ae954a90ce76 P e823b55004f64499978fb94e96d2f89f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e823b55004f64499978fb94e96d2f89f, State: Initialized, Role: FOLLOWER
I20260926 15:06:18.835753 25717 heartbeater.cc:503] Master 127.9.123.126:45713 was elected leader, sending a full tablet report...
I20260926 15:06:18.836261 25737 ts_tablet_manager.cc:1428] T 83e480b3b9134583b967ae954a90ce76 P 2566a89ae54442eca72b9b8add797384: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260926 15:06:18.836228 25735 consensus_queue.cc:260] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } } peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } }
I20260926 15:06:18.836228 25736 consensus_queue.cc:260] T 83e480b3b9134583b967ae954a90ce76 P e823b55004f64499978fb94e96d2f89f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } } peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } }
I20260926 15:06:18.837414 25583 heartbeater.cc:503] Master 127.9.123.126:45713 was elected leader, sending a full tablet report...
I20260926 15:06:18.838122 25736 ts_tablet_manager.cc:1428] T 83e480b3b9134583b967ae954a90ce76 P e823b55004f64499978fb94e96d2f89f: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260926 15:06:18.839742 25735 ts_tablet_manager.cc:1428] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
W20260926 15:06:18.923774 25450 tablet.cc:2369] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:06:18.931623 25742 raft_consensus.cc:482] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:06:18.931929 25742 raft_consensus.cc:504] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } } peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } }
I20260926 15:06:18.933775 25742 leader_election.cc:288] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e823b55004f64499978fb94e96d2f89f (127.9.123.66:42499), 2566a89ae54442eca72b9b8add797384 (127.9.123.67:37657)
I20260926 15:06:18.941664 25731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:18.934153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:54425 (local address 127.9.123.66:42499)
0926 15:06:18.934404 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:18.934410 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:18.934936 (+ 526us) server_negotiation.cc:407] Connection header received
0926 15:06:18.935228 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:18.935234 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:18.935317 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:18.935404 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:06:18.936106 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.937020 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.939480 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.940555 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.940703 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:18.940762 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:18.941247 (+ 485us) server_negotiation.cc:299] Negotiation successful
0926 15:06:18.941398 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 15:06:18.941713 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:18.934149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:42499 (local address 127.9.123.65:54425)
0926 15:06:18.934814 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:18.934831 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:06:18.935054 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:06:18.935535 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:06:18.935560 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:06:18.935590 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:06:18.935967 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:18.935979 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.937178 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.937183 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:18.939270 (+ 2087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:18.939286 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.940801 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.940807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:18.940989 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:18.941026 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 15:06:18.941185 (+ 159us) client_negotiation.cc:240] Negotiation successful
0926 15:06:18.941349 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"mutex_wait_us":23,"thread_start_us":365,"threads_started":1}
I20260926 15:06:18.942116 25733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:18.935243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:41823 (local address 127.9.123.67:37657)
0926 15:06:18.935448 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:18.935454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:18.935703 (+ 249us) server_negotiation.cc:407] Connection header received
0926 15:06:18.935875 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:18.935880 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:18.935955 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:18.936063 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:06:18.936615 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.937637 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.940100 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.941089 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.941271 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:18.941337 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:18.941741 (+ 404us) server_negotiation.cc:299] Negotiation successful
0926 15:06:18.941880 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260926 15:06:18.942639 25745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:18.934941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:37657 (local address 127.9.123.65:41823)
0926 15:06:18.935592 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:18.935609 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:06:18.935745 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:06:18.936096 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:06:18.936101 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:06:18.936125 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:06:18.936463 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:18.936475 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.937791 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.937796 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:18.939674 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:18.939711 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.941335 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.941341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:18.941516 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:18.941553 (+ 37us) client_negotiation.cc:769] Sending connection context
0926 15:06:18.941707 (+ 154us) client_negotiation.cc:240] Negotiation successful
0926 15:06:18.942213 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"mutex_wait_us":63,"thread_start_us":336,"threads_started":1}
I20260926 15:06:18.942845 25538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83e480b3b9134583b967ae954a90ce76" candidate_uuid: "3bd7598679d84b62bbb2a0082717b051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e823b55004f64499978fb94e96d2f89f" is_pre_election: true
I20260926 15:06:18.943398 25538 raft_consensus.cc:2454] T 83e480b3b9134583b967ae954a90ce76 P e823b55004f64499978fb94e96d2f89f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bd7598679d84b62bbb2a0082717b051 in term 0.
I20260926 15:06:18.943693 25672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83e480b3b9134583b967ae954a90ce76" candidate_uuid: "3bd7598679d84b62bbb2a0082717b051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2566a89ae54442eca72b9b8add797384" is_pre_election: true
I20260926 15:06:18.944186 25337 leader_election.cc:302] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bd7598679d84b62bbb2a0082717b051, e823b55004f64499978fb94e96d2f89f; no voters:
I20260926 15:06:18.944255 25672 raft_consensus.cc:2454] T 83e480b3b9134583b967ae954a90ce76 P 2566a89ae54442eca72b9b8add797384 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bd7598679d84b62bbb2a0082717b051 in term 0.
I20260926 15:06:18.944694 25742 raft_consensus.cc:2781] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:06:18.944891 25742 raft_consensus.cc:482] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:06:18.945013 25742 raft_consensus.cc:3037] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051 [term 0 FOLLOWER]: Advancing to term 1
W20260926 15:06:18.946133 25718 tablet.cc:2369] T 83e480b3b9134583b967ae954a90ce76 P 2566a89ae54442eca72b9b8add797384: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:06:18.947753 25742 raft_consensus.cc:504] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } } peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } }
I20260926 15:06:18.948374 25742 leader_election.cc:288] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051 [CANDIDATE]: Term 1 election: Requested vote from peers e823b55004f64499978fb94e96d2f89f (127.9.123.66:42499), 2566a89ae54442eca72b9b8add797384 (127.9.123.67:37657)
I20260926 15:06:18.949092 25672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83e480b3b9134583b967ae954a90ce76" candidate_uuid: "3bd7598679d84b62bbb2a0082717b051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2566a89ae54442eca72b9b8add797384"
I20260926 15:06:18.949090 25538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83e480b3b9134583b967ae954a90ce76" candidate_uuid: "3bd7598679d84b62bbb2a0082717b051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e823b55004f64499978fb94e96d2f89f"
I20260926 15:06:18.949371 25672 raft_consensus.cc:3037] T 83e480b3b9134583b967ae954a90ce76 P 2566a89ae54442eca72b9b8add797384 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:06:18.949383 25538 raft_consensus.cc:3037] T 83e480b3b9134583b967ae954a90ce76 P e823b55004f64499978fb94e96d2f89f [term 0 FOLLOWER]: Advancing to term 1
W20260926 15:06:18.950022 25584 tablet.cc:2369] T 83e480b3b9134583b967ae954a90ce76 P e823b55004f64499978fb94e96d2f89f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:06:18.952638 25672 raft_consensus.cc:2454] T 83e480b3b9134583b967ae954a90ce76 P 2566a89ae54442eca72b9b8add797384 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bd7598679d84b62bbb2a0082717b051 in term 1.
I20260926 15:06:18.952638 25538 raft_consensus.cc:2454] T 83e480b3b9134583b967ae954a90ce76 P e823b55004f64499978fb94e96d2f89f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bd7598679d84b62bbb2a0082717b051 in term 1.
I20260926 15:06:18.953187 25337 leader_election.cc:302] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2566a89ae54442eca72b9b8add797384, 3bd7598679d84b62bbb2a0082717b051; no voters:
I20260926 15:06:18.953548 25742 raft_consensus.cc:2781] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:06:18.954007 25742 raft_consensus.cc:686] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051 [term 1 LEADER]: Becoming Leader. State: Replica: 3bd7598679d84b62bbb2a0082717b051, State: Running, Role: LEADER
I20260926 15:06:18.954530 25742 consensus_queue.cc:237] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } } peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } }
I20260926 15:06:18.959117 25260 catalog_manager.cc:5495] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bd7598679d84b62bbb2a0082717b051 (127.9.123.65). New cstate: current_term: 1 leader_uuid: "3bd7598679d84b62bbb2a0082717b051" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } health_report { overall_health: UNKNOWN } } }
I20260926 15:06:18.987481 25732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:18.981999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37496 (local address 127.9.123.65:45717)
0926 15:06:18.982202 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:18.982207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:18.982231 (+ 24us) server_negotiation.cc:407] Connection header received
0926 15:06:18.982494 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:18.982498 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:18.982580 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:18.982727 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:06:18.983080 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.983826 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.985250 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:18.985662 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:18.985770 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:18.986333 (+ 563us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:06:18.986868 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:06:18.987002 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:18.987111 (+ 109us) server_negotiation.cc:299] Negotiation successful
0926 15:06:18.987199 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 15:06:18.996181 25538 raft_consensus.cc:1261] T 83e480b3b9134583b967ae954a90ce76 P e823b55004f64499978fb94e96d2f89f [term 1 FOLLOWER]: Refusing update from remote peer 3bd7598679d84b62bbb2a0082717b051: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:06:18.996178 25672 raft_consensus.cc:1261] T 83e480b3b9134583b967ae954a90ce76 P 2566a89ae54442eca72b9b8add797384 [term 1 FOLLOWER]: Refusing update from remote peer 3bd7598679d84b62bbb2a0082717b051: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:06:18.997138 25742 consensus_queue.cc:1035] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:06:18.997447 25746 consensus_queue.cc:1035] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:06:19.024291 25260 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:06:19.024606 25260 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:06:19.028841 25260 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15: Sending DeleteTablet for 3 replicas of tablet 83e480b3b9134583b967ae954a90ce76
I20260926 15:06:19.030331 25518 tablet_service.cc:1514] Processing DeleteTablet for tablet 83e480b3b9134583b967ae954a90ce76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:06:19 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:44872
I20260926 15:06:19.030328 25384 tablet_service.cc:1514] Processing DeleteTablet for tablet 83e480b3b9134583b967ae954a90ce76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:06:19 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:37480
I20260926 15:06:19.030491 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 25321
I20260926 15:06:19.031430 25652 tablet_service.cc:1514] Processing DeleteTablet for tablet 83e480b3b9134583b967ae954a90ce76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:06:19 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:49798
I20260926 15:06:19.039759 25762 tablet_replica.cc:330] T 83e480b3b9134583b967ae954a90ce76 P 2566a89ae54442eca72b9b8add797384: stopping tablet replica
I20260926 15:06:19.041061 25762 raft_consensus.cc:2229] T 83e480b3b9134583b967ae954a90ce76 P 2566a89ae54442eca72b9b8add797384 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:06:19.042495 25762 raft_consensus.cc:2258] T 83e480b3b9134583b967ae954a90ce76 P 2566a89ae54442eca72b9b8add797384 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:06:19.048003 25759 tablet_replica.cc:330] T 83e480b3b9134583b967ae954a90ce76 P e823b55004f64499978fb94e96d2f89f: stopping tablet replica
I20260926 15:06:19.048499 25759 raft_consensus.cc:2229] T 83e480b3b9134583b967ae954a90ce76 P e823b55004f64499978fb94e96d2f89f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:06:19.048900 25762 ts_tablet_manager.cc:1905] T 83e480b3b9134583b967ae954a90ce76 P 2566a89ae54442eca72b9b8add797384: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:06:19.049044 25759 raft_consensus.cc:2258] T 83e480b3b9134583b967ae954a90ce76 P e823b55004f64499978fb94e96d2f89f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:06:19.049957 25760 tablet_replica.cc:330] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051: stopping tablet replica
I20260926 15:06:19.050421 25760 raft_consensus.cc:2229] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051 [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:06:19.051160 25760 raft_consensus.cc:2258] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:06:19.056926 25759 ts_tablet_manager.cc:1905] T 83e480b3b9134583b967ae954a90ce76 P e823b55004f64499978fb94e96d2f89f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:06:19.057708 25762 ts_tablet_manager.cc:1918] T 83e480b3b9134583b967ae954a90ce76 P 2566a89ae54442eca72b9b8add797384: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:06:19.057968 25762 log.cc:1196] T 83e480b3b9134583b967ae954a90ce76 P 2566a89ae54442eca72b9b8add797384: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/83e480b3b9134583b967ae954a90ce76
I20260926 15:06:19.058524 25762 ts_tablet_manager.cc:1939] T 83e480b3b9134583b967ae954a90ce76 P 2566a89ae54442eca72b9b8add797384: Deleting consensus metadata
I20260926 15:06:19.060698 25244 catalog_manager.cc:4841] TS 2566a89ae54442eca72b9b8add797384 (127.9.123.67:37657): tablet 83e480b3b9134583b967ae954a90ce76 (table test-table [id=8908949602a947debeb0f6056077d7fe]) successfully deleted
I20260926 15:06:19.061261 25760 ts_tablet_manager.cc:1905] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:06:19.067267 25760 ts_tablet_manager.cc:1918] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:06:19.067466 25760 log.cc:1196] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/wal/wals/83e480b3b9134583b967ae954a90ce76
I20260926 15:06:19.068063 25760 ts_tablet_manager.cc:1939] T 83e480b3b9134583b967ae954a90ce76 P 3bd7598679d84b62bbb2a0082717b051: Deleting consensus metadata
I20260926 15:06:19.068395 25759 ts_tablet_manager.cc:1918] T 83e480b3b9134583b967ae954a90ce76 P e823b55004f64499978fb94e96d2f89f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:06:19.068634 25759 log.cc:1196] T 83e480b3b9134583b967ae954a90ce76 P e823b55004f64499978fb94e96d2f89f: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/83e480b3b9134583b967ae954a90ce76
I20260926 15:06:19.069228 25759 ts_tablet_manager.cc:1939] T 83e480b3b9134583b967ae954a90ce76 P e823b55004f64499978fb94e96d2f89f: Deleting consensus metadata
I20260926 15:06:19.070130 25246 catalog_manager.cc:4841] TS 3bd7598679d84b62bbb2a0082717b051 (127.9.123.65:45717): tablet 83e480b3b9134583b967ae954a90ce76 (table test-table [id=8908949602a947debeb0f6056077d7fe]) successfully deleted
I20260926 15:06:19.071027 25244 catalog_manager.cc:4841] TS e823b55004f64499978fb94e96d2f89f (127.9.123.66:42499): tablet 83e480b3b9134583b967ae954a90ce76 (table test-table [id=8908949602a947debeb0f6056077d7fe]) successfully deleted
Sep 26 15:06:19 dist-test-slave-7153 krb5kdc[25215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.65@KRBTEST.COM
I20260926 15:06:19.083351 25732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:19.031657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37498 (local address 127.9.123.65:45717)
0926 15:06:19.031834 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:19.031841 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:19.043575 (+ 11734us) server_negotiation.cc:407] Connection header received
0926 15:06:19.043766 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:19.043776 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:19.043883 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:19.044006 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:06:19.046222 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:19.047480 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:19.049603 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:19.050375 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:19.050524 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:19.075270 (+ 24746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:06:19.075288 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:06:19.075308 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:06:19.075366 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:06:19.079919 (+ 4553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:19.080876 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:19.080891 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:19.080894 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:19.081130 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:19.081523 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:19.081528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:19.081531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:19.082053 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:06:19.082386 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:19.082670 (+ 284us) server_negotiation.cc:299] Negotiation successful
0926 15:06:19.082945 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 15:06:19.084178 25444 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:06:19.190682 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 25321
I20260926 15:06:19.208729 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 25455
Sep 26 15:06:19 dist-test-slave-7153 krb5kdc[25215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.66@KRBTEST.COM
I20260926 15:06:19.226821 25731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:19.209522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44884 (local address 127.9.123.66:42499)
0926 15:06:19.209700 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:19.209707 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:19.209725 (+ 18us) server_negotiation.cc:407] Connection header received
0926 15:06:19.209929 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:19.209933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:19.210022 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:19.210122 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:06:19.211772 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:19.212914 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:19.214338 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:19.214963 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:19.215084 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:19.220644 (+ 5560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:06:19.220661 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:06:19.220677 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:06:19.220725 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:06:19.224005 (+ 3280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:19.224825 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:19.224833 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:19.224836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:19.224978 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:19.225351 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:19.225354 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:19.225356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:19.225785 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:06:19.226026 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:19.226271 (+ 245us) server_negotiation.cc:299] Negotiation successful
0926 15:06:19.226455 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 15:06:19.227720 25578 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:06:19.320923 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 25455
I20260926 15:06:19.334991 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 25589
Sep 26 15:06:19 dist-test-slave-7153 krb5kdc[25215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.67@KRBTEST.COM
I20260926 15:06:19.350756 25733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:19.335780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49800 (local address 127.9.123.67:37657)
0926 15:06:19.335967 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:19.335973 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:19.335993 (+ 20us) server_negotiation.cc:407] Connection header received
0926 15:06:19.336175 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:19.336181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:19.336255 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:19.336336 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:06:19.337780 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:19.338504 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:19.339880 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:19.340297 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:19.340410 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:19.344958 (+ 4548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:06:19.344976 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:06:19.344993 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:06:19.345035 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:06:19.347820 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:19.348572 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:19.348581 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:19.348583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:19.348735 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:19.349174 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:19.349179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:19.349182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:19.349565 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:06:19.349845 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:19.350179 (+ 334us) server_negotiation.cc:299] Negotiation successful
0926 15:06:19.350395 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 15:06:19.351402 25712 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:06:19.440300 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 25589
I20260926 15:06:19.453908 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 25228
I20260926 15:06:19.455086 25290 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:06:19.568794 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 25228
I20260926 15:06:39.587205 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:45713
--webserver_interface=127.9.123.126
--webserver_port=41349
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:33955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:45713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:06:39.922475 25767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:06:39.922799 25767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:06:39.922930 25767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:06:39.931556 25767 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 15:06:39.931682 25767 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 15:06:39.931749 25767 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 15:06:39.931815 25767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:06:39.931888 25767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:06:39.931986 25767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:06:39.932046 25767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:06:39.932125 25767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:06:39.942636 25767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:33955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:45713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:45713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=41349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:06:39.944025 25767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:06:39.945509 25767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:06:39.968297 25773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:06:39.968295 25774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:06:39.968946 25776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:06:39.969499 25767 server_base.cc:1034] running on GCE node
I20260926 15:06:39.970571 25767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:06:39.972369 25767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:06:39.973630 25767 hybrid_clock.cc:648] HybridClock initialized: now 1790435199973533 us; error 62 us; skew 500 ppm
I20260926 15:06:39.974107 25767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:06:39 dist-test-slave-7153 krb5kdc[25215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435199, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:06:39.977923 25767 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:06:39.979820 25767 webserver.cc:460] Webserver started at http://127.9.123.126:41349/ using document root <none> and password file <none>
I20260926 15:06:39.980808 25767 fs_manager.cc:359] Metadata directory not provided
I20260926 15:06:39.980949 25767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:06:39.987555 25767 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260926 15:06:39.991091 25783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:06:39.992687 25767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260926 15:06:39.992899 25767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "51190f416ba240538a331f103c888f15"
format_stamp: "Formatted at 2026-09-26 15:06:16 on dist-test-slave-7153"
server_key: "ee27df8aec007c1b2827e5e464497254"
server_key_iv: "11f4ff4135b2044dcf5af14d87bbe19b"
server_key_version: "encryptionkey@0"
I20260926 15:06:39.994112 25767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:06:40.014405 25767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:06:40.019170 25767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:06:40.019490 25767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:06:40.037956 25767 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:45713
I20260926 15:06:40.037978 25834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:45713 every 8 connection(s)
I20260926 15:06:40.039515 25767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:06:40.040769 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 25767
I20260926 15:06:40.041769 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:45717
--local_ip_for_outbound_sockets=127.9.123.65
--tserver_master_addrs=127.9.123.126:45713
--webserver_port=35783
--webserver_interface=127.9.123.65
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:33955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:06:40.049292 25835 sys_catalog.cc:263] Verifying existing consensus state
I20260926 15:06:40.053242 25835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15: Bootstrap starting.
I20260926 15:06:40.073524 25835 log.cc:824] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15: Log is configured to *not* fsync() on all Append() calls
I20260926 15:06:40.086887 25835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 15:06:40.087468 25835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15: Bootstrap complete.
I20260926 15:06:40.097568 25835 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51190f416ba240538a331f103c888f15" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45713 } }
I20260926 15:06:40.098651 25835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51190f416ba240538a331f103c888f15, State: Initialized, Role: FOLLOWER
I20260926 15:06:40.099376 25835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51190f416ba240538a331f103c888f15" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45713 } }
I20260926 15:06:40.099639 25835 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:06:40.099759 25835 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:06:40.099927 25835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 1 FOLLOWER]: Advancing to term 2
I20260926 15:06:40.104060 25835 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51190f416ba240538a331f103c888f15" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45713 } }
I20260926 15:06:40.104631 25835 leader_election.cc:302] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51190f416ba240538a331f103c888f15; no voters:
I20260926 15:06:40.105273 25835 leader_election.cc:288] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260926 15:06:40.105417 25839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 2 FOLLOWER]: Leader election won for term 2
I20260926 15:06:40.106655 25839 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 2 LEADER]: Becoming Leader. State: Replica: 51190f416ba240538a331f103c888f15, State: Running, Role: LEADER
I20260926 15:06:40.107254 25839 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51190f416ba240538a331f103c888f15" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45713 } }
I20260926 15:06:40.107764 25835 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:06:40.109759 25841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51190f416ba240538a331f103c888f15. Latest consensus state: current_term: 2 leader_uuid: "51190f416ba240538a331f103c888f15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51190f416ba240538a331f103c888f15" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45713 } } }
I20260926 15:06:40.110229 25841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [sys.catalog]: This master's current role is: LEADER
I20260926 15:06:40.110848 25840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "51190f416ba240538a331f103c888f15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51190f416ba240538a331f103c888f15" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45713 } } }
I20260926 15:06:40.111155 25840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [sys.catalog]: This master's current role is: LEADER
I20260926 15:06:40.120464 25846 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:06:40.129679 25846 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:06:40.131502 25846 catalog_manager.cc:1259] Loaded cluster ID: 4c182619a59440af9eb564a4b411c7e0
I20260926 15:06:40.131711 25846 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:06:40.136155 25846 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:06:40.137904 25846 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15: Loaded TSK: 0
I20260926 15:06:40.146548 25856 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 15:06:40.168541 25846 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15: Generated new TSK 1
I20260926 15:06:40.169008 25846 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260926 15:06:40.409229 25837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:06:40.409533 25837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:06:40.409677 25837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:06:40.417193 25837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:06:40.417320 25837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:06:40.417471 25837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:06:40.428012 25837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:33955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:45717
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=35783
--tserver_master_addrs=127.9.123.126:45713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:06:40.429555 25837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:06:40.431270 25837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:06:40.447685 25863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:06:40.447896 25862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:06:40.448084 25865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:06:40.448505 25837 server_base.cc:1034] running on GCE node
I20260926 15:06:40.449178 25837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:06:40.450217 25837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:06:40.451543 25837 hybrid_clock.cc:648] HybridClock initialized: now 1790435200451476 us; error 63 us; skew 500 ppm
I20260926 15:06:40.451893 25837 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:06:40 dist-test-slave-7153 krb5kdc[25215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435200, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:06:40.455360 25837 init.cc:377] Logged in from keytab as kudu/127.9.123.65@KRBTEST.COM (short username kudu)
I20260926 15:06:40.456939 25837 webserver.cc:460] Webserver started at http://127.9.123.65:35783/ using document root <none> and password file <none>
I20260926 15:06:40.457692 25837 fs_manager.cc:359] Metadata directory not provided
I20260926 15:06:40.457825 25837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:06:40.463538 25837 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260926 15:06:40.466686 25872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:06:40.468223 25837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:06:40.468382 25837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "3bd7598679d84b62bbb2a0082717b051"
format_stamp: "Formatted at 2026-09-26 15:06:17 on dist-test-slave-7153"
server_key: "a7836fa31b6310506ee5d532b937c04b"
server_key_iv: "afd5550bca1e9f1400b925636321072f"
server_key_version: "encryptionkey@0"
I20260926 15:06:40.469457 25837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:06:40.493796 25837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:06:40.498404 25837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:06:40.498673 25837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:06:40.499774 25837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:06:40.501466 25837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:06:40.501583 25837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:06:40.501698 25837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:06:40.501765 25837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:06:40.532981 25837 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:45717
I20260926 15:06:40.533059 25984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:45717 every 8 connection(s)
I20260926 15:06:40.534451 25837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:06:40.539453 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 25837
I20260926 15:06:40.540233 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:42499
--local_ip_for_outbound_sockets=127.9.123.66
--tserver_master_addrs=127.9.123.126:45713
--webserver_port=39617
--webserver_interface=127.9.123.66
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:33955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 15:06:40 dist-test-slave-7153 krb5kdc[25215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435200, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:06:40.564590 25987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:40.537299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:46747 (local address 127.9.123.126:45713)
0926 15:06:40.537918 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:40.537937 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:40.538550 (+ 613us) server_negotiation.cc:407] Connection header received
0926 15:06:40.541480 (+ 2930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:40.541496 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:40.541920 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:40.542267 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:06:40.544504 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:40.546086 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:40.548540 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:40.549725 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:40.549943 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:40.555008 (+ 5065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:06:40.555019 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:06:40.555037 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:06:40.555086 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:06:40.558533 (+ 3447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:40.559842 (+ 1309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:40.559852 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:40.559859 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:40.560089 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:40.560794 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:40.560799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:40.560802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:40.561473 (+ 671us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:06:40.561658 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:40.563156 (+ 1498us) server_negotiation.cc:299] Negotiation successful
0926 15:06:40.563568 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":256,"threads_started":1}
I20260926 15:06:40.564940 25988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:40.537577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.65:46747)
0926 15:06:40.538372 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:40.538410 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 15:06:40.540778 (+ 2368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:06:40.542498 (+ 1720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:06:40.542508 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:06:40.543258 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:06:40.544205 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:40.544239 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:40.546305 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:40.546310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:40.548312 (+ 2002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:40.548331 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:40.550093 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:40.550104 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:40.550347 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:40.551191 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:06:40.551217 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:06:40.554550 (+ 3333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:06:40.558792 (+ 4242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:06:40.558804 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:06:40.558824 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:06:40.559612 (+ 788us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:06:40.560290 (+ 678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:06:40.560296 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:06:40.560299 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:06:40.560614 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:06:40.561691 (+ 1077us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:06:40.561701 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:06:40.562393 (+ 692us) client_negotiation.cc:769] Sending connection context
0926 15:06:40.562777 (+ 384us) client_negotiation.cc:240] Negotiation successful
0926 15:06:40.563529 (+ 752us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":338,"threads_started":1}
I20260926 15:06:40.568732 25985 heartbeater.cc:347] Connected to a master server at 127.9.123.126:45713
I20260926 15:06:40.569070 25985 heartbeater.cc:464] Registering TS with master...
I20260926 15:06:40.569900 25985 heartbeater.cc:511] Master 127.9.123.126:45713 requested a full tablet report, sending...
I20260926 15:06:40.572762 25800 ts_manager.cc:194] Registered new tserver with Master: 3bd7598679d84b62bbb2a0082717b051 (127.9.123.65:45717)
I20260926 15:06:40.575100 25800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.65@KRBTEST.COM'} at 127.9.123.65:46747
W20260926 15:06:40.882587 25989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:06:40.882879 25989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:06:40.882974 25989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:06:40.890281 25989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:06:40.890416 25989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:06:40.890532 25989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:06:40.901192 25989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:33955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:42499
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=39617
--tserver_master_addrs=127.9.123.126:45713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:06:40.902491 25989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:06:40.903841 25989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:06:40.917830 25994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:06:40.917861 25995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:06:40.917936 25997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:06:40.918690 25989 server_base.cc:1034] running on GCE node
I20260926 15:06:40.919399 25989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:06:40.920421 25989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:06:40.921653 25989 hybrid_clock.cc:648] HybridClock initialized: now 1790435200921596 us; error 38 us; skew 500 ppm
I20260926 15:06:40.921998 25989 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:06:40 dist-test-slave-7153 krb5kdc[25215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435200, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:06:40.925403 25989 init.cc:377] Logged in from keytab as kudu/127.9.123.66@KRBTEST.COM (short username kudu)
I20260926 15:06:40.926992 25989 webserver.cc:460] Webserver started at http://127.9.123.66:39617/ using document root <none> and password file <none>
I20260926 15:06:40.927742 25989 fs_manager.cc:359] Metadata directory not provided
I20260926 15:06:40.927882 25989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:06:40.934504 25989 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 15:06:40.937422 26004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:06:40.938851 25989 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:06:40.939013 25989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "e823b55004f64499978fb94e96d2f89f"
format_stamp: "Formatted at 2026-09-26 15:06:18 on dist-test-slave-7153"
server_key: "4df59d307b65f0a0a439a903c9d9fac2"
server_key_iv: "790190b5d108337d13f192be265264a2"
server_key_version: "encryptionkey@0"
I20260926 15:06:40.940011 25989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:06:40.961321 25989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:06:40.966022 25989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:06:40.966302 25989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:06:40.967429 25989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:06:40.968910 25989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:06:40.969017 25989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:06:40.969154 25989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:06:40.969225 25989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:06:40.999615 25989 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:42499
I20260926 15:06:40.999683 26116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:42499 every 8 connection(s)
I20260926 15:06:41.001072 25989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:06:41.007742 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 25989
I20260926 15:06:41.008386 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:37657
--local_ip_for_outbound_sockets=127.9.123.67
--tserver_master_addrs=127.9.123.126:45713
--webserver_port=37819
--webserver_interface=127.9.123.67
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:33955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 26 15:06:41 dist-test-slave-7153 krb5kdc[25215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435200, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:06:41.021749 25987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:41.003531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:47521 (local address 127.9.123.126:45713)
0926 15:06:41.003755 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:41.003761 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:41.005006 (+ 1245us) server_negotiation.cc:407] Connection header received
0926 15:06:41.006973 (+ 1967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:41.006977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:41.007065 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:41.007158 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:06:41.009031 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.010075 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.011750 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.012318 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.012451 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:41.016071 (+ 3620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:06:41.016077 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:06:41.016081 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:06:41.016118 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:06:41.018656 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:41.019413 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:41.019418 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:41.019420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:41.019551 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:41.019983 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:41.019987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:41.019989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:41.020269 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:06:41.020424 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:41.021153 (+ 729us) server_negotiation.cc:299] Negotiation successful
0926 15:06:41.021413 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 15:06:41.022786 26119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:41.004088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.66:47521)
0926 15:06:41.004821 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:41.004867 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 15:06:41.006656 (+ 1789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:06:41.007476 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:06:41.007489 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:06:41.008150 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:06:41.008833 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:41.008860 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.010269 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.010273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:41.011602 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:41.011615 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.012540 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.012545 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:41.012750 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:41.013467 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:06:41.013495 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:06:41.015768 (+ 2273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:06:41.018793 (+ 3025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:06:41.018803 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:06:41.018824 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:06:41.019277 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:06:41.019660 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:06:41.019665 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:06:41.019669 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:06:41.019875 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:06:41.020394 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:06:41.020403 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:06:41.020765 (+ 362us) client_negotiation.cc:769] Sending connection context
0926 15:06:41.021031 (+ 266us) client_negotiation.cc:240] Negotiation successful
0926 15:06:41.021657 (+ 626us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":285,"threads_started":1}
I20260926 15:06:41.024318 26117 heartbeater.cc:347] Connected to a master server at 127.9.123.126:45713
I20260926 15:06:41.024674 26117 heartbeater.cc:464] Registering TS with master...
I20260926 15:06:41.025629 26117 heartbeater.cc:511] Master 127.9.123.126:45713 requested a full tablet report, sending...
I20260926 15:06:41.027273 25800 ts_manager.cc:194] Registered new tserver with Master: e823b55004f64499978fb94e96d2f89f (127.9.123.66:42499)
I20260926 15:06:41.028144 25800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.66@KRBTEST.COM'} at 127.9.123.66:47521
W20260926 15:06:41.340468 26120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:06:41.340747 26120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:06:41.340866 26120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:06:41.348204 26120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:06:41.348320 26120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:06:41.348554 26120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:06:41.358608 26120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:33955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:37657
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=37819
--tserver_master_addrs=127.9.123.126:45713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:06:41.359895 26120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:06:41.361227 26120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:06:41.374298 26125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:06:41.374229 26128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:06:41.374261 26126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:06:41.375527 26120 server_base.cc:1034] running on GCE node
I20260926 15:06:41.376179 26120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:06:41.377224 26120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:06:41.378435 26120 hybrid_clock.cc:648] HybridClock initialized: now 1790435201378390 us; error 36 us; skew 500 ppm
I20260926 15:06:41.378743 26120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:06:41 dist-test-slave-7153 krb5kdc[25215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435201, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:06:41.381887 26120 init.cc:377] Logged in from keytab as kudu/127.9.123.67@KRBTEST.COM (short username kudu)
I20260926 15:06:41.383385 26120 webserver.cc:460] Webserver started at http://127.9.123.67:37819/ using document root <none> and password file <none>
I20260926 15:06:41.384058 26120 fs_manager.cc:359] Metadata directory not provided
I20260926 15:06:41.384171 26120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:06:41.389873 26120 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260926 15:06:41.393155 26135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:06:41.394765 26120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 15:06:41.394960 26120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "2566a89ae54442eca72b9b8add797384"
format_stamp: "Formatted at 2026-09-26 15:06:18 on dist-test-slave-7153"
server_key: "0a63ffd4a54320d83bb914d3e888a06e"
server_key_iv: "243c624b9f50610ab8f3304a3ec28830"
server_key_version: "encryptionkey@0"
I20260926 15:06:41.395990 26120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:06:41.415601 26120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:06:41.420234 26120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:06:41.420503 26120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:06:41.421619 26120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:06:41.423070 26120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:06:41.423173 26120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:06:41.423286 26120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:06:41.423358 26120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:06:41.454792 26120 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:37657
I20260926 15:06:41.454849 26247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:37657 every 8 connection(s)
I20260926 15:06:41.456178 26120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:06:41.462276 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 26120
Sep 26 15:06:41 dist-test-slave-7153 krb5kdc[25215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435201, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:06:41.479292 26252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:41.463408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34022 (local address 127.9.123.126:45713)
0926 15:06:41.464145 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:41.464151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:41.464168 (+ 17us) server_negotiation.cc:407] Connection header received
0926 15:06:41.464219 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:41.464222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:41.464275 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:41.464359 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:06:41.465947 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.467013 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.468408 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.468926 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.469050 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:41.471848 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:06:41.471855 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:06:41.471860 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:06:41.471904 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:06:41.475488 (+ 3584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:41.476476 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:41.476482 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:41.476486 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:41.476709 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:41.477300 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:41.477307 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:41.477311 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:41.477831 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:06:41.478104 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:41.478580 (+ 476us) server_negotiation.cc:299] Negotiation successful
0926 15:06:41.478776 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":626,"spinlock_wait_cycles":11904,"thread_start_us":498,"threads_started":1}
I20260926 15:06:41.479822 25987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:41.458785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:55337 (local address 127.9.123.126:45713)
0926 15:06:41.458988 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:41.458993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:41.460216 (+ 1223us) server_negotiation.cc:407] Connection header received
0926 15:06:41.462164 (+ 1948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:41.462169 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:41.462244 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:41.462334 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:06:41.464434 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.465255 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.467335 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.467766 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.467870 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:41.471548 (+ 3678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:06:41.471552 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:06:41.471556 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:06:41.471586 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:06:41.475440 (+ 3854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:41.476476 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:41.476482 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:41.476486 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:41.476689 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:41.477303 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:41.477307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:41.477311 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:41.477831 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:06:41.477995 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:41.478983 (+ 988us) server_negotiation.cc:299] Negotiation successful
0926 15:06:41.479194 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":78,"server-negotiator.queue_time_us":68,"spinlock_wait_cycles":24704}
I20260926 15:06:41.480764 26250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:41.459256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.67:55337)
0926 15:06:41.460032 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:41.460079 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 15:06:41.461808 (+ 1729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:06:41.462730 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:06:41.462743 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:06:41.463475 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:06:41.464260 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:41.464281 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.465415 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.465419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:41.467217 (+ 1798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:41.467229 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.467996 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.468014 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:41.468236 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:41.468907 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:06:41.468944 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:06:41.471336 (+ 2392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:06:41.475635 (+ 4299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:06:41.475646 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:06:41.475666 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:06:41.476333 (+ 667us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:06:41.476858 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:06:41.476864 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:06:41.476867 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:06:41.477190 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:06:41.478021 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:06:41.478031 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:06:41.478520 (+ 489us) client_negotiation.cc:769] Sending connection context
0926 15:06:41.478849 (+ 329us) client_negotiation.cc:240] Negotiation successful
0926 15:06:41.479405 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":312,"threads_started":1}
I20260926 15:06:41.482160 26248 heartbeater.cc:347] Connected to a master server at 127.9.123.126:45713
I20260926 15:06:41.482453 26248 heartbeater.cc:464] Registering TS with master...
I20260926 15:06:41.483147 26248 heartbeater.cc:511] Master 127.9.123.126:45713 requested a full tablet report, sending...
I20260926 15:06:41.484452 25800 ts_manager.cc:194] Registered new tserver with Master: 2566a89ae54442eca72b9b8add797384 (127.9.123.67:37657)
I20260926 15:06:41.485327 25800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.67@KRBTEST.COM'} at 127.9.123.67:55337
I20260926 15:06:41.494809 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 15:06:41.501835 25987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:41.495865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34034 (local address 127.9.123.126:45713)
0926 15:06:41.496096 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:41.496103 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:41.496458 (+ 355us) server_negotiation.cc:407] Connection header received
0926 15:06:41.496677 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:41.496681 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:41.496833 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:41.496978 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:06:41.497442 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.498268 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.499800 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.500225 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.500340 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:41.500911 (+ 571us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:06:41.501154 (+ 243us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0926 15:06:41.501579 (+ 425us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34034: token expired
Metrics: {"server-negotiator.queue_time_us":71}
W20260926 15:06:41.501972 25987 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34034: token expired
W20260926 15:06:41.502573 25721 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.9.123.126:45713)
I20260926 15:06:41.515897 25987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:41.503085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34042 (local address 127.9.123.126:45713)
0926 15:06:41.503252 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:41.503257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:41.503273 (+ 16us) server_negotiation.cc:407] Connection header received
0926 15:06:41.503439 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:41.503445 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:41.503508 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:41.503585 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:06:41.504918 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.505705 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.507115 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.507523 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.507632 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:41.510355 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:06:41.510361 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:06:41.510365 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:06:41.510411 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:06:41.513222 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:41.513933 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:41.513939 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:41.513943 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:41.514083 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:41.514439 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:41.514445 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:41.514448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:41.514797 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:06:41.515032 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:41.515338 (+ 306us) server_negotiation.cc:299] Negotiation successful
0926 15:06:41.515560 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 15:06:41.519488 25800 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:06:41.527695 25800 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:06:41.563721 26261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:41.542885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57878 (local address 127.9.123.67:37657)
0926 15:06:41.556366 (+ 13481us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:41.556376 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:41.556423 (+ 47us) server_negotiation.cc:407] Connection header received
0926 15:06:41.556526 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:41.556535 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:41.556804 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:41.557021 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:06:41.557963 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.559123 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.561260 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.562507 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.562639 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:41.562880 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:41.563218 (+ 338us) server_negotiation.cc:299] Negotiation successful
0926 15:06:41.563380 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13295,"thread_start_us":13015,"threads_started":1}
I20260926 15:06:41.563860 26262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:41.542563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:37657 (local address 127.0.0.1:57878)
0926 15:06:41.551713 (+ 9150us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:41.551740 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 15:06:41.551903 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:06:41.557329 (+ 5426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:06:41.557335 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:06:41.557362 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:06:41.557700 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:41.557712 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.559280 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.559285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:41.560991 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:41.561004 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.562754 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.562761 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:41.562992 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:41.563034 (+ 42us) client_negotiation.cc:769] Sending connection context
0926 15:06:41.563151 (+ 117us) client_negotiation.cc:240] Negotiation successful
0926 15:06:41.563364 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8988,"mutex_wait_us":5,"thread_start_us":14571,"threads_started":1}
I20260926 15:06:41.563884 26258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:41.541183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:42499 (local address 127.0.0.1:48912)
0926 15:06:41.541959 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:41.541998 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 15:06:41.542133 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:06:41.555320 (+ 13187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:06:41.555329 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:06:41.555383 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:06:41.555915 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:41.555930 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.558015 (+ 2085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.558020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:41.560743 (+ 2723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:41.560756 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.562795 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.562801 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:41.563008 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:41.563041 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 15:06:41.563136 (+ 95us) client_negotiation.cc:240] Negotiation successful
0926 15:06:41.563372 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":42,"thread_start_us":436,"threads_started":1}
I20260926 15:06:41.564910 26259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:41.541358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48912 (local address 127.9.123.66:42499)
0926 15:06:41.554250 (+ 12892us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:41.554261 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:41.554297 (+ 36us) server_negotiation.cc:407] Connection header received
0926 15:06:41.554428 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:41.554438 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:41.554773 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:41.556181 (+ 1408us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:06:41.556545 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.557871 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.561083 (+ 3212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.562549 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.564153 (+ 1604us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:41.564360 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:41.564427 (+ 67us) server_negotiation.cc:299] Negotiation successful
0926 15:06:41.564593 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12685,"thread_start_us":13601,"threads_started":1}
I20260926 15:06:41.568125 26260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:41.543396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:45717 (local address 127.0.0.1:60504)
0926 15:06:41.556921 (+ 13525us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:41.556940 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:06:41.557051 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:06:41.561193 (+ 4142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:06:41.561197 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:06:41.561223 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:06:41.561547 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:41.561558 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.563276 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.563281 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:41.565701 (+ 2420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:41.565714 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.567352 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.567359 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:41.567526 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:41.567559 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 15:06:41.567654 (+ 95us) client_negotiation.cc:240] Negotiation successful
0926 15:06:41.567783 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13372,"thread_start_us":7990,"threads_started":1}
I20260926 15:06:41.568181 26263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:41.544107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60504 (local address 127.9.123.65:45717)
0926 15:06:41.560564 (+ 16457us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:41.560573 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:41.560604 (+ 31us) server_negotiation.cc:407] Connection header received
0926 15:06:41.560705 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:41.560714 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:41.560988 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:41.561218 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:06:41.561690 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.563132 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.565860 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.567063 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.567196 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:41.567386 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:41.567733 (+ 347us) server_negotiation.cc:299] Negotiation successful
0926 15:06:41.567914 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16274,"thread_start_us":17265,"threads_started":1}
I20260926 15:06:41.567734 26183 tablet_service.cc:1467] Processing CreateTablet for tablet 68da91940810430790b78a8f0a2b6730 (DEFAULT_TABLE table=test-table [id=a3b2bc986efe48a187659a10b239ac43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:06:41.570031 26183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68da91940810430790b78a8f0a2b6730. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:06:41.571955 26052 tablet_service.cc:1467] Processing CreateTablet for tablet 68da91940810430790b78a8f0a2b6730 (DEFAULT_TABLE table=test-table [id=a3b2bc986efe48a187659a10b239ac43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:06:41.572333 25920 tablet_service.cc:1467] Processing CreateTablet for tablet 68da91940810430790b78a8f0a2b6730 (DEFAULT_TABLE table=test-table [id=a3b2bc986efe48a187659a10b239ac43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:06:41.573182 26052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68da91940810430790b78a8f0a2b6730. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:06:41.574277 25920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68da91940810430790b78a8f0a2b6730. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:06:41.579113 25985 heartbeater.cc:503] Master 127.9.123.126:45713 was elected leader, sending a full tablet report...
I20260926 15:06:41.585759 26264 tablet_bootstrap.cc:492] T 68da91940810430790b78a8f0a2b6730 P 2566a89ae54442eca72b9b8add797384: Bootstrap starting.
I20260926 15:06:41.588847 26266 tablet_bootstrap.cc:492] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051: Bootstrap starting.
I20260926 15:06:41.590318 26264 tablet_bootstrap.cc:654] T 68da91940810430790b78a8f0a2b6730 P 2566a89ae54442eca72b9b8add797384: Neither blocks nor log segments found. Creating new log.
I20260926 15:06:41.591687 26266 tablet_bootstrap.cc:654] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051: Neither blocks nor log segments found. Creating new log.
I20260926 15:06:41.591737 26264 log.cc:824] T 68da91940810430790b78a8f0a2b6730 P 2566a89ae54442eca72b9b8add797384: Log is configured to *not* fsync() on all Append() calls
I20260926 15:06:41.592479 26265 tablet_bootstrap.cc:492] T 68da91940810430790b78a8f0a2b6730 P e823b55004f64499978fb94e96d2f89f: Bootstrap starting.
I20260926 15:06:41.592847 26266 log.cc:824] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051: Log is configured to *not* fsync() on all Append() calls
I20260926 15:06:41.595367 26264 tablet_bootstrap.cc:492] T 68da91940810430790b78a8f0a2b6730 P 2566a89ae54442eca72b9b8add797384: No bootstrap required, opened a new log
I20260926 15:06:41.595654 26264 ts_tablet_manager.cc:1397] T 68da91940810430790b78a8f0a2b6730 P 2566a89ae54442eca72b9b8add797384: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260926 15:06:41.596351 26265 tablet_bootstrap.cc:654] T 68da91940810430790b78a8f0a2b6730 P e823b55004f64499978fb94e96d2f89f: Neither blocks nor log segments found. Creating new log.
I20260926 15:06:41.597401 26266 tablet_bootstrap.cc:492] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051: No bootstrap required, opened a new log
I20260926 15:06:41.597373 26265 log.cc:824] T 68da91940810430790b78a8f0a2b6730 P e823b55004f64499978fb94e96d2f89f: Log is configured to *not* fsync() on all Append() calls
I20260926 15:06:41.597685 26266 ts_tablet_manager.cc:1397] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260926 15:06:41.606501 26265 tablet_bootstrap.cc:492] T 68da91940810430790b78a8f0a2b6730 P e823b55004f64499978fb94e96d2f89f: No bootstrap required, opened a new log
I20260926 15:06:41.606353 26264 raft_consensus.cc:348] T 68da91940810430790b78a8f0a2b6730 P 2566a89ae54442eca72b9b8add797384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } } peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } }
I20260926 15:06:41.606815 26264 raft_consensus.cc:374] T 68da91940810430790b78a8f0a2b6730 P 2566a89ae54442eca72b9b8add797384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:06:41.606932 26264 raft_consensus.cc:729] T 68da91940810430790b78a8f0a2b6730 P 2566a89ae54442eca72b9b8add797384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2566a89ae54442eca72b9b8add797384, State: Initialized, Role: FOLLOWER
I20260926 15:06:41.607339 26265 ts_tablet_manager.cc:1397] T 68da91940810430790b78a8f0a2b6730 P e823b55004f64499978fb94e96d2f89f: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.008s
I20260926 15:06:41.607700 26264 consensus_queue.cc:260] T 68da91940810430790b78a8f0a2b6730 P 2566a89ae54442eca72b9b8add797384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } } peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } }
I20260926 15:06:41.607983 26266 raft_consensus.cc:348] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } } peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } }
I20260926 15:06:41.608546 26266 raft_consensus.cc:374] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:06:41.608655 26266 raft_consensus.cc:729] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd7598679d84b62bbb2a0082717b051, State: Initialized, Role: FOLLOWER
I20260926 15:06:41.609117 26248 heartbeater.cc:503] Master 127.9.123.126:45713 was elected leader, sending a full tablet report...
I20260926 15:06:41.609422 26266 consensus_queue.cc:260] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } } peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } }
I20260926 15:06:41.611244 26266 ts_tablet_manager.cc:1428] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260926 15:06:41.611511 26264 ts_tablet_manager.cc:1428] T 68da91940810430790b78a8f0a2b6730 P 2566a89ae54442eca72b9b8add797384: Time spent starting tablet: real 0.016s user 0.009s sys 0.006s
I20260926 15:06:41.617498 26265 raft_consensus.cc:348] T 68da91940810430790b78a8f0a2b6730 P e823b55004f64499978fb94e96d2f89f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } } peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } }
I20260926 15:06:41.617866 26265 raft_consensus.cc:374] T 68da91940810430790b78a8f0a2b6730 P e823b55004f64499978fb94e96d2f89f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:06:41.617969 26265 raft_consensus.cc:729] T 68da91940810430790b78a8f0a2b6730 P e823b55004f64499978fb94e96d2f89f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e823b55004f64499978fb94e96d2f89f, State: Initialized, Role: FOLLOWER
I20260926 15:06:41.618600 26265 consensus_queue.cc:260] T 68da91940810430790b78a8f0a2b6730 P e823b55004f64499978fb94e96d2f89f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } } peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } }
I20260926 15:06:41.619601 26117 heartbeater.cc:503] Master 127.9.123.126:45713 was elected leader, sending a full tablet report...
I20260926 15:06:41.620074 26265 ts_tablet_manager.cc:1428] T 68da91940810430790b78a8f0a2b6730 P e823b55004f64499978fb94e96d2f89f: Time spent starting tablet: real 0.013s user 0.008s sys 0.003s
I20260926 15:06:41.669591 26271 raft_consensus.cc:482] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:06:41.669910 26271 raft_consensus.cc:504] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } } peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } }
I20260926 15:06:41.671660 26271 leader_election.cc:288] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e823b55004f64499978fb94e96d2f89f (127.9.123.66:42499), 2566a89ae54442eca72b9b8add797384 (127.9.123.67:37657)
I20260926 15:06:41.678367 26261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:41.672961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:43509 (local address 127.9.123.67:37657)
0926 15:06:41.673189 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:41.673195 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:41.673519 (+ 324us) server_negotiation.cc:407] Connection header received
0926 15:06:41.673677 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:41.673683 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:41.673748 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:41.673821 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:06:41.674307 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.675006 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.676699 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.677530 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.677673 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:41.677742 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:41.678008 (+ 266us) server_negotiation.cc:299] Negotiation successful
0926 15:06:41.678131 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 15:06:41.678377 26259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:41.672055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:43931 (local address 127.9.123.66:42499)
0926 15:06:41.672231 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:41.672236 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:41.672837 (+ 601us) server_negotiation.cc:407] Connection header received
0926 15:06:41.673125 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:41.673131 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:41.673199 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:41.673284 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:06:41.673883 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.674636 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.676606 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.677445 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.677606 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:41.677681 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:41.677950 (+ 269us) server_negotiation.cc:299] Negotiation successful
0926 15:06:41.678069 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 15:06:41.678457 26274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:41.672848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:37657 (local address 127.9.123.65:43509)
0926 15:06:41.673413 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:41.673440 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0926 15:06:41.673555 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:06:41.673855 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:06:41.673859 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:06:41.673882 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:06:41.674168 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:41.674177 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.675129 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.675137 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:41.676555 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:41.676567 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.677741 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.677747 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:41.677872 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:41.677894 (+ 22us) client_negotiation.cc:769] Sending connection context
0926 15:06:41.677973 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 15:06:41.678094 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":341,"threads_started":1}
I20260926 15:06:41.678495 26273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:41.672055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:42499 (local address 127.9.123.65:43931)
0926 15:06:41.672737 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:41.672756 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:06:41.672903 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:06:41.673326 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:06:41.673332 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:06:41.673360 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:06:41.673741 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:41.673755 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.674770 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.674775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:41.676427 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:06:41.676440 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.677642 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.677648 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:06:41.677809 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:41.677833 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 15:06:41.677913 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 15:06:41.678057 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":75,"thread_start_us":327,"threads_started":1}
I20260926 15:06:41.679577 26203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68da91940810430790b78a8f0a2b6730" candidate_uuid: "3bd7598679d84b62bbb2a0082717b051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2566a89ae54442eca72b9b8add797384" is_pre_election: true
I20260926 15:06:41.679553 26072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68da91940810430790b78a8f0a2b6730" candidate_uuid: "3bd7598679d84b62bbb2a0082717b051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e823b55004f64499978fb94e96d2f89f" is_pre_election: true
I20260926 15:06:41.680099 26072 raft_consensus.cc:2454] T 68da91940810430790b78a8f0a2b6730 P e823b55004f64499978fb94e96d2f89f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bd7598679d84b62bbb2a0082717b051 in term 0.
I20260926 15:06:41.680100 26203 raft_consensus.cc:2454] T 68da91940810430790b78a8f0a2b6730 P 2566a89ae54442eca72b9b8add797384 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bd7598679d84b62bbb2a0082717b051 in term 0.
I20260926 15:06:41.680722 25873 leader_election.cc:302] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bd7598679d84b62bbb2a0082717b051, e823b55004f64499978fb94e96d2f89f; no voters:
I20260926 15:06:41.681071 26271 raft_consensus.cc:2781] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:06:41.681237 26271 raft_consensus.cc:482] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:06:41.681331 26271 raft_consensus.cc:3037] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:06:41.683411 26271 raft_consensus.cc:504] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } } peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } }
I20260926 15:06:41.683960 26271 leader_election.cc:288] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [CANDIDATE]: Term 1 election: Requested vote from peers e823b55004f64499978fb94e96d2f89f (127.9.123.66:42499), 2566a89ae54442eca72b9b8add797384 (127.9.123.67:37657)
I20260926 15:06:41.684386 26072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68da91940810430790b78a8f0a2b6730" candidate_uuid: "3bd7598679d84b62bbb2a0082717b051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e823b55004f64499978fb94e96d2f89f"
I20260926 15:06:41.684471 26203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68da91940810430790b78a8f0a2b6730" candidate_uuid: "3bd7598679d84b62bbb2a0082717b051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2566a89ae54442eca72b9b8add797384"
I20260926 15:06:41.684571 26072 raft_consensus.cc:3037] T 68da91940810430790b78a8f0a2b6730 P e823b55004f64499978fb94e96d2f89f [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:06:41.684674 26203 raft_consensus.cc:3037] T 68da91940810430790b78a8f0a2b6730 P 2566a89ae54442eca72b9b8add797384 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:06:41.686818 26072 raft_consensus.cc:2454] T 68da91940810430790b78a8f0a2b6730 P e823b55004f64499978fb94e96d2f89f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bd7598679d84b62bbb2a0082717b051 in term 1.
I20260926 15:06:41.686816 26203 raft_consensus.cc:2454] T 68da91940810430790b78a8f0a2b6730 P 2566a89ae54442eca72b9b8add797384 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bd7598679d84b62bbb2a0082717b051 in term 1.
I20260926 15:06:41.687218 25873 leader_election.cc:302] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bd7598679d84b62bbb2a0082717b051, e823b55004f64499978fb94e96d2f89f; no voters:
I20260926 15:06:41.687486 26271 raft_consensus.cc:2781] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:06:41.687918 26271 raft_consensus.cc:686] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 1 LEADER]: Becoming Leader. State: Replica: 3bd7598679d84b62bbb2a0082717b051, State: Running, Role: LEADER
I20260926 15:06:41.688628 26271 consensus_queue.cc:237] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } } peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } }
I20260926 15:06:41.693446 25800 catalog_manager.cc:5495] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bd7598679d84b62bbb2a0082717b051 (127.9.123.65). New cstate: current_term: 1 leader_uuid: "3bd7598679d84b62bbb2a0082717b051" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } health_report { overall_health: HEALTHY } } }
W20260926 15:06:41.707759 26249 tablet.cc:2369] T 68da91940810430790b78a8f0a2b6730 P 2566a89ae54442eca72b9b8add797384: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:06:41.708707 26263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:41.703319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60506 (local address 127.9.123.65:45717)
0926 15:06:41.703513 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:41.703519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:41.703537 (+ 18us) server_negotiation.cc:407] Connection header received
0926 15:06:41.703762 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:41.703766 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:41.703844 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:41.703978 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:06:41.704316 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.705032 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.706469 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.706887 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.707005 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:41.707560 (+ 555us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:06:41.708121 (+ 561us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:06:41.708259 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:41.708360 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 15:06:41.708452 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 15:06:41.717388 26072 raft_consensus.cc:1261] T 68da91940810430790b78a8f0a2b6730 P e823b55004f64499978fb94e96d2f89f [term 1 FOLLOWER]: Refusing update from remote peer 3bd7598679d84b62bbb2a0082717b051: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:06:41.717386 26203 raft_consensus.cc:1261] T 68da91940810430790b78a8f0a2b6730 P 2566a89ae54442eca72b9b8add797384 [term 1 FOLLOWER]: Refusing update from remote peer 3bd7598679d84b62bbb2a0082717b051: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:06:41.718405 26275 consensus_queue.cc:1035] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:06:41.718803 26271 consensus_queue.cc:1035] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:06:41.732187 26278 mvcc.cc:205] Tried to move back new op lower bound from 7333622586224119808 to 7333622586120343552. Current Snapshot: MvccSnapshot[applied={T|T < 7333622586224119808}]
I20260926 15:06:41.749413 25800 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:06:41.749694 25800 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:06:41.753450 25800 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15: Sending DeleteTablet for 3 replicas of tablet 68da91940810430790b78a8f0a2b6730
I20260926 15:06:41.754778 26183 tablet_service.cc:1514] Processing DeleteTablet for tablet 68da91940810430790b78a8f0a2b6730 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:06:41 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:57878
I20260926 15:06:41.754801 25920 tablet_service.cc:1514] Processing DeleteTablet for tablet 68da91940810430790b78a8f0a2b6730 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:06:41 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:60504
I20260926 15:06:41.755512 26052 tablet_service.cc:1514] Processing DeleteTablet for tablet 68da91940810430790b78a8f0a2b6730 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:06:41 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:48912
I20260926 15:06:41.755645 26288 tablet_replica.cc:330] T 68da91940810430790b78a8f0a2b6730 P 2566a89ae54442eca72b9b8add797384: stopping tablet replica
I20260926 15:06:41.756232 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 25837
I20260926 15:06:41.756197 26288 raft_consensus.cc:2229] T 68da91940810430790b78a8f0a2b6730 P 2566a89ae54442eca72b9b8add797384 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:06:41.756835 26288 raft_consensus.cc:2258] T 68da91940810430790b78a8f0a2b6730 P 2566a89ae54442eca72b9b8add797384 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:06:41.759243 26288 ts_tablet_manager.cc:1905] T 68da91940810430790b78a8f0a2b6730 P 2566a89ae54442eca72b9b8add797384: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:06:41.766338 26288 ts_tablet_manager.cc:1918] T 68da91940810430790b78a8f0a2b6730 P 2566a89ae54442eca72b9b8add797384: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:06:41.767088 26290 tablet_replica.cc:330] T 68da91940810430790b78a8f0a2b6730 P e823b55004f64499978fb94e96d2f89f: stopping tablet replica
I20260926 15:06:41.767645 26290 raft_consensus.cc:2229] T 68da91940810430790b78a8f0a2b6730 P e823b55004f64499978fb94e96d2f89f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:06:41.768283 26290 raft_consensus.cc:2258] T 68da91940810430790b78a8f0a2b6730 P e823b55004f64499978fb94e96d2f89f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:06:41.768739 26288 log.cc:1196] T 68da91940810430790b78a8f0a2b6730 P 2566a89ae54442eca72b9b8add797384: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/68da91940810430790b78a8f0a2b6730
I20260926 15:06:41.769469 26288 ts_tablet_manager.cc:1939] T 68da91940810430790b78a8f0a2b6730 P 2566a89ae54442eca72b9b8add797384: Deleting consensus metadata
I20260926 15:06:41.771018 26290 ts_tablet_manager.cc:1905] T 68da91940810430790b78a8f0a2b6730 P e823b55004f64499978fb94e96d2f89f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:06:41.776501 26263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:41.757706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60514 (local address 127.9.123.65:45717)
0926 15:06:41.757862 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:41.757868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:41.757884 (+ 16us) server_negotiation.cc:407] Connection header received
0926 15:06:41.757941 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:41.757946 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:41.758018 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:41.758113 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:06:41.760063 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.761284 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.763222 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.763935 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.764060 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:41.767775 (+ 3715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:06:41.767793 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:06:41.767814 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:06:41.767866 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:06:41.771999 (+ 4133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:41.772998 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:41.773007 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:41.773011 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:41.773260 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:41.773735 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:41.773740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:41.773744 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:41.774267 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:06:41.774562 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:41.775808 (+ 1246us) server_negotiation.cc:299] Negotiation successful
0926 15:06:41.776079 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 15:06:41.777328 25980 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:06:41.779109 25784 catalog_manager.cc:4841] TS 2566a89ae54442eca72b9b8add797384 (127.9.123.67:37657): tablet 68da91940810430790b78a8f0a2b6730 (table test-table [id=a3b2bc986efe48a187659a10b239ac43]) successfully deleted
I20260926 15:06:41.779194 26290 ts_tablet_manager.cc:1918] T 68da91940810430790b78a8f0a2b6730 P e823b55004f64499978fb94e96d2f89f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:06:41.779356 26290 log.cc:1196] T 68da91940810430790b78a8f0a2b6730 P e823b55004f64499978fb94e96d2f89f: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/68da91940810430790b78a8f0a2b6730
I20260926 15:06:41.779907 26290 ts_tablet_manager.cc:1939] T 68da91940810430790b78a8f0a2b6730 P e823b55004f64499978fb94e96d2f89f: Deleting consensus metadata
I20260926 15:06:41.781766 25784 catalog_manager.cc:4841] TS e823b55004f64499978fb94e96d2f89f (127.9.123.66:42499): tablet 68da91940810430790b78a8f0a2b6730 (table test-table [id=a3b2bc986efe48a187659a10b239ac43]) successfully deleted
I20260926 15:06:41.875680 26289 tablet_replica.cc:330] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051: stopping tablet replica
I20260926 15:06:41.876006 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 25837
W20260926 15:06:41.891206 25786 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.123.65:45717 (error 108)
I20260926 15:06:41.891518 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 25989
I20260926 15:06:41.893159 26260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:41.892661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:45717 (local address 127.0.0.1:60516)
0926 15:06:41.892851 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:41.892966 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:45717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260926 15:06:41.893396 25786 catalog_manager.cc:4584] TS 3bd7598679d84b62bbb2a0082717b051 (127.9.123.65:45717): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 68da91940810430790b78a8f0a2b6730: Network error: Client connection negotiation failed: client connection to 127.9.123.65:45717: connect: Connection refused (error 111)
I20260926 15:06:41.905886 26259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:41.892246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48914 (local address 127.9.123.66:42499)
0926 15:06:41.892418 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:41.892422 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:41.892440 (+ 18us) server_negotiation.cc:407] Connection header received
0926 15:06:41.892621 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:41.892636 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:41.892721 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:41.892824 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:06:41.894321 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.895130 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.896486 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:41.896951 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:41.897059 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:41.899990 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:06:41.900006 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:06:41.900023 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:06:41.900063 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:06:41.902989 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:41.903762 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:41.903770 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:41.903772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:41.903950 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:41.904267 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:41.904271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:41.904273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:41.904667 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:06:41.904890 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:41.905440 (+ 550us) server_negotiation.cc:299] Negotiation successful
0926 15:06:41.905628 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 15:06:41.906682 26112 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:06:41.947384 26260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:41.946803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:45717 (local address 127.0.0.1:60532)
0926 15:06:41.947061 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:41.947193 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:45717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260926 15:06:41.948593 26260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:41.948158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:45717 (local address 127.0.0.1:60536)
0926 15:06:41.948319 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:41.948423 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:45717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260926 15:06:41.997254 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 25989
I20260926 15:06:42.009378 26260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:42.008750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:45717 (local address 127.0.0.1:60542)
0926 15:06:42.009003 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:42.009147 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:45717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260926 15:06:42.010516 26260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:42.010126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:45717 (local address 127.0.0.1:60558)
0926 15:06:42.010285 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:42.010375 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:45717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260926 15:06:42.011518 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 26120
I20260926 15:06:42.025539 26261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:42.012186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57892 (local address 127.9.123.67:37657)
0926 15:06:42.012366 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 15:06:42.012372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:06:42.012393 (+ 21us) server_negotiation.cc:407] Connection header received
0926 15:06:42.012577 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:06:42.012584 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:06:42.012675 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:06:42.012775 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:06:42.014217 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:42.014964 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:42.016244 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:06:42.016657 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:06:42.016774 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:06:42.019659 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:06:42.019676 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:06:42.019692 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:06:42.019731 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:06:42.022600 (+ 2869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:42.023344 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:42.023352 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:42.023354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:42.023519 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:06:42.023918 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:06:42.023924 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:06:42.023927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:06:42.024294 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:06:42.024583 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0926 15:06:42.024894 (+ 311us) server_negotiation.cc:299] Negotiation successful
0926 15:06:42.025161 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 15:06:42.026237 26243 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:06:42.091166 26260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:42.090555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:45717 (local address 127.0.0.1:60570)
0926 15:06:42.090815 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:42.090949 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:45717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260926 15:06:42.092397 26260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:06:42.091940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:45717 (local address 127.0.0.1:60582)
0926 15:06:42.092155 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0926 15:06:42.092245 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:45717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260926 15:06:42.114811 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 26120
I20260926 15:06:42.128392 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 25767
I20260926 15:06:42.129705 25830 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:06:42.243381 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 25767
I20260926 15:07:02.261650 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:45713
--webserver_interface=127.9.123.126
--webserver_port=41349
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:33955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:45713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:07:02.587957 26296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:02.588274 26296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:02.588394 26296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:02.595881 26296 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260926 15:07:02.595988 26296 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260926 15:07:02.596028 26296 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260926 15:07:02.596101 26296 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:07:02.596180 26296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:02.596262 26296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:02.596325 26296 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:07:02.596392 26296 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:07:02.606720 26296 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:33955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:45713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:45713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=41349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:02.607985 26296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:02.609332 26296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:02.621031 26304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:02.621121 26301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:02.621071 26302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:02.621935 26296 server_base.cc:1034] running on GCE node
I20260926 15:07:02.622732 26296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:02.624255 26296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:02.625449 26296 hybrid_clock.cc:648] HybridClock initialized: now 1790435222625378 us; error 47 us; skew 500 ppm
I20260926 15:07:02.625765 26296 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:02 dist-test-slave-7153 krb5kdc[25215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435222, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:02.628961 26296 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:07:02.630506 26296 webserver.cc:460] Webserver started at http://127.9.123.126:41349/ using document root <none> and password file <none>
I20260926 15:07:02.631207 26296 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:02.631314 26296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:02.636919 26296 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260926 15:07:02.639789 26311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:02.641172 26296 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 15:07:02.641368 26296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "51190f416ba240538a331f103c888f15"
format_stamp: "Formatted at 2026-09-26 15:06:16 on dist-test-slave-7153"
server_key: "ee27df8aec007c1b2827e5e464497254"
server_key_iv: "11f4ff4135b2044dcf5af14d87bbe19b"
server_key_version: "encryptionkey@0"
I20260926 15:07:02.642346 26296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:02.659476 26296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:02.663594 26296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:02.663872 26296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:02.681811 26296 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:45713
I20260926 15:07:02.681857 26362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:45713 every 8 connection(s)
I20260926 15:07:02.683275 26296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:07:02.684903 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 26296
I20260926 15:07:02.685920 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:45717
--local_ip_for_outbound_sockets=127.9.123.65
--tserver_master_addrs=127.9.123.126:45713
--webserver_port=35783
--webserver_interface=127.9.123.65
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:33955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:07:02.692430 26363 sys_catalog.cc:263] Verifying existing consensus state
I20260926 15:07:02.696497 26363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15: Bootstrap starting.
I20260926 15:07:02.717026 26363 log.cc:824] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:02.734256 26363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 15:07:02.734810 26363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15: Bootstrap complete.
I20260926 15:07:02.743790 26363 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51190f416ba240538a331f103c888f15" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45713 } }
I20260926 15:07:02.744817 26363 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51190f416ba240538a331f103c888f15, State: Initialized, Role: FOLLOWER
I20260926 15:07:02.745492 26363 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51190f416ba240538a331f103c888f15" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45713 } }
I20260926 15:07:02.745736 26363 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:07:02.745882 26363 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:07:02.746057 26363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 2 FOLLOWER]: Advancing to term 3
I20260926 15:07:02.750083 26363 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51190f416ba240538a331f103c888f15" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45713 } }
I20260926 15:07:02.750584 26363 leader_election.cc:302] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51190f416ba240538a331f103c888f15; no voters:
I20260926 15:07:02.751286 26363 leader_election.cc:288] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260926 15:07:02.751406 26367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 3 FOLLOWER]: Leader election won for term 3
I20260926 15:07:02.752471 26367 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [term 3 LEADER]: Becoming Leader. State: Replica: 51190f416ba240538a331f103c888f15, State: Running, Role: LEADER
I20260926 15:07:02.753134 26367 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51190f416ba240538a331f103c888f15" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45713 } }
I20260926 15:07:02.753468 26363 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:07:02.755676 26369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51190f416ba240538a331f103c888f15. Latest consensus state: current_term: 3 leader_uuid: "51190f416ba240538a331f103c888f15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51190f416ba240538a331f103c888f15" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45713 } } }
I20260926 15:07:02.755718 26368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "51190f416ba240538a331f103c888f15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51190f416ba240538a331f103c888f15" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45713 } } }
I20260926 15:07:02.756045 26369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:02.756090 26368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15 [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:02.766319 26374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:07:02.780625 26374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:07:02.782335 26374 catalog_manager.cc:1259] Loaded cluster ID: 4c182619a59440af9eb564a4b411c7e0
I20260926 15:07:02.782519 26374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:07:02.786731 26374 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:07:02.788336 26374 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15: Loaded TSK: 0
I20260926 15:07:02.788558 26374 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 51190f416ba240538a331f103c888f15: Loaded TSK: 1
F20260926 15:07:02.789664 26374 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f7623f219ad google::LogMessage::Fail() at ??:0
@ 0x7f7623f2488a google::LogMessage::SendToLog() at ??:0
@ 0x7f7623f214de google::LogMessage::Flush() at ??:0
@ 0x7f7623f22a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f7627c857d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f7627c7473e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f76399501c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f763994f783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f76399db2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f76399db0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f7639a255e9 std::function<>::operator()() at ??:0
@ 0x7f7639953cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f7639951c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f76399d9610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f76399d9439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f7626b6d345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f7626b7091c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f7626b70799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f7626b4beaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f7629e256db start_thread at ??:0
@ 0x7f7622dbb61f clone at ??:0
W20260926 15:07:03.089591 26365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:03.089888 26365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:03.090013 26365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:03.097576 26365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:03.097718 26365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:03.097879 26365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:07:03.107929 26365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:33955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:45717
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=35783
--tserver_master_addrs=127.9.123.126:45713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:03.109277 26365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:03.110603 26365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:03.123592 26411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:03.123652 26410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:03.123631 26413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:03.125016 26365 server_base.cc:1034] running on GCE node
I20260926 15:07:03.125705 26365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:03.126663 26365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:03.127866 26365 hybrid_clock.cc:648] HybridClock initialized: now 1790435223127815 us; error 36 us; skew 500 ppm
I20260926 15:07:03.128176 26365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:03 dist-test-slave-7153 krb5kdc[25215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435223, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:03.131417 26365 init.cc:377] Logged in from keytab as kudu/127.9.123.65@KRBTEST.COM (short username kudu)
I20260926 15:07:03.132925 26365 webserver.cc:460] Webserver started at http://127.9.123.65:35783/ using document root <none> and password file <none>
I20260926 15:07:03.133637 26365 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:03.133744 26365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:03.139264 26365 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260926 15:07:03.142351 26420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:03.143812 26365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 15:07:03.143955 26365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "3bd7598679d84b62bbb2a0082717b051"
format_stamp: "Formatted at 2026-09-26 15:06:17 on dist-test-slave-7153"
server_key: "a7836fa31b6310506ee5d532b937c04b"
server_key_iv: "afd5550bca1e9f1400b925636321072f"
server_key_version: "encryptionkey@0"
I20260926 15:07:03.144884 26365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:03.172094 26365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:03.176501 26365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:03.176759 26365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:03.177841 26365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:07:03.179701 26427 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260926 15:07:03.185523 26365 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260926 15:07:03.185653 26365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260926 15:07:03.185761 26365 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260926 15:07:03.189085 26365 ts_tablet_manager.cc:610] Registered 1 tablets
I20260926 15:07:03.189232 26365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.004s sys 0.000s
I20260926 15:07:03.189625 26427 tablet_bootstrap.cc:492] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051: Bootstrap starting.
I20260926 15:07:03.217126 26427 log.cc:824] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:03.224856 26365 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:45717
I20260926 15:07:03.224905 26534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:45717 every 8 connection(s)
I20260926 15:07:03.226581 26365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:07:03.232844 26427 tablet_bootstrap.cc:492] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260926 15:07:03.233669 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 26365
I20260926 15:07:03.233743 26427 tablet_bootstrap.cc:492] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051: Bootstrap complete.
I20260926 15:07:03.234318 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:42499
--local_ip_for_outbound_sockets=127.9.123.66
--tserver_master_addrs=127.9.123.126:45713
--webserver_port=39617
--webserver_interface=127.9.123.66
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:33955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:07:03.235704 26427 ts_tablet_manager.cc:1397] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051: Time spent bootstrapping tablet: real 0.046s user 0.031s sys 0.012s
I20260926 15:07:03.247681 26427 raft_consensus.cc:348] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } } peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } }
I20260926 15:07:03.249881 26427 raft_consensus.cc:729] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd7598679d84b62bbb2a0082717b051, State: Initialized, Role: FOLLOWER
I20260926 15:07:03.250831 26427 consensus_queue.cc:260] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } } peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } }
I20260926 15:07:03.254076 26537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:03.240303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.65:54839)
0926 15:07:03.252084 (+ 11781us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:03.252418 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11382,"threads_started":1}
W20260926 15:07:03.255540 26535 heartbeater.cc:644] Failed to heartbeat to 127.9.123.126:45713 (0 consecutive failures): Network error: Failed to ping master at 127.9.123.126:45713: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
I20260926 15:07:03.255924 26427 ts_tablet_manager.cc:1428] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051: Time spent starting tablet: real 0.020s user 0.010s sys 0.012s
I20260926 15:07:03.256644 26537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:03.256221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.65:45485)
0926 15:07:03.256412 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:03.256500 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260926 15:07:03.257828 26537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:03.257362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.65:43863)
0926 15:07:03.257542 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:03.257633 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260926 15:07:03.258095 26535 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 15:07:03.574020 26538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:03.574313 26538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:03.574409 26538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:03.581795 26538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:03.581915 26538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:03.582026 26538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:07:03.592072 26538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:33955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:42499
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=39617
--tserver_master_addrs=127.9.123.126:45713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:03.593407 26538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:03.594699 26538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:03.607453 26545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:03.607545 26547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:03.607502 26544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:03.608500 26538 server_base.cc:1034] running on GCE node
I20260926 15:07:03.609238 26538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:03.610201 26538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:03.611411 26538 hybrid_clock.cc:648] HybridClock initialized: now 1790435223611359 us; error 46 us; skew 500 ppm
I20260926 15:07:03.611743 26538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:03 dist-test-slave-7153 krb5kdc[25215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435223, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:03.614892 26538 init.cc:377] Logged in from keytab as kudu/127.9.123.66@KRBTEST.COM (short username kudu)
I20260926 15:07:03.616302 26538 webserver.cc:460] Webserver started at http://127.9.123.66:39617/ using document root <none> and password file <none>
I20260926 15:07:03.616946 26538 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:03.617044 26538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:03.622594 26538 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260926 15:07:03.625413 26554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:03.626946 26538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 15:07:03.627125 26538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "e823b55004f64499978fb94e96d2f89f"
format_stamp: "Formatted at 2026-09-26 15:06:18 on dist-test-slave-7153"
server_key: "4df59d307b65f0a0a439a903c9d9fac2"
server_key_iv: "790190b5d108337d13f192be265264a2"
server_key_version: "encryptionkey@0"
I20260926 15:07:03.628193 26538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:03.642820 26538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:03.647285 26538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:03.647549 26538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:03.648545 26538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:07:03.650007 26538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:07:03.650101 26538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:03.650213 26538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:07:03.650279 26538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:03.681229 26538 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:42499
I20260926 15:07:03.681268 26666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:42499 every 8 connection(s)
I20260926 15:07:03.682694 26538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:07:03.685621 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 26538
I20260926 15:07:03.686246 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:37657
--local_ip_for_outbound_sockets=127.9.123.67
--tserver_master_addrs=127.9.123.126:45713
--webserver_port=37819
--webserver_interface=127.9.123.67
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:33955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:07:03.687813 26669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:03.685728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.66:33201)
0926 15:07:03.686572 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:03.686874 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":323,"threads_started":1}
W20260926 15:07:03.688437 26667 heartbeater.cc:644] Failed to heartbeat to 127.9.123.126:45713 (0 consecutive failures): Network error: Failed to ping master at 127.9.123.126:45713: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
I20260926 15:07:03.689371 26669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:03.688998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.66:44737)
0926 15:07:03.689184 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:03.689264 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260926 15:07:03.690198 26669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:03.689885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.66:58809)
0926 15:07:03.690007 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:03.690080 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260926 15:07:03.690428 26667 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260926 15:07:04.017355 26670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:04.017663 26670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:04.017783 26670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:04.025307 26670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:04.025434 26670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:04.025573 26670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:07:04.035884 26670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:33955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:37657
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=37819
--tserver_master_addrs=127.9.123.126:45713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:04.037264 26670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:04.038686 26670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:04.051954 26676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:04.052047 26675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:04.052063 26678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:04.053050 26670 server_base.cc:1034] running on GCE node
I20260926 15:07:04.053792 26670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:04.054849 26670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:04.056068 26670 hybrid_clock.cc:648] HybridClock initialized: now 1790435224055991 us; error 64 us; skew 500 ppm
I20260926 15:07:04.056394 26670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:04 dist-test-slave-7153 krb5kdc[25215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435224, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:04.059629 26670 init.cc:377] Logged in from keytab as kudu/127.9.123.67@KRBTEST.COM (short username kudu)
I20260926 15:07:04.061275 26670 webserver.cc:460] Webserver started at http://127.9.123.67:37819/ using document root <none> and password file <none>
I20260926 15:07:04.061986 26670 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:04.062094 26670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:04.067907 26670 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260926 15:07:04.070963 26685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:04.072453 26670 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 15:07:04.072634 26670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "2566a89ae54442eca72b9b8add797384"
format_stamp: "Formatted at 2026-09-26 15:06:18 on dist-test-slave-7153"
server_key: "0a63ffd4a54320d83bb914d3e888a06e"
server_key_iv: "243c624b9f50610ab8f3304a3ec28830"
server_key_version: "encryptionkey@0"
I20260926 15:07:04.073908 26670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:04.089800 26670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:04.094220 26670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:04.094491 26670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:04.095517 26670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:07:04.096928 26670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:07:04.097025 26670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:04.097150 26670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:07:04.097229 26670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:04.128263 26670 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:37657
I20260926 15:07:04.128332 26797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:37657 every 8 connection(s)
I20260926 15:07:04.129755 26670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:07:04.134739 26800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:04.132789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.67:38155)
0926 15:07:04.133587 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:04.133841 (+ 254us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":292,"threads_started":1}
W20260926 15:07:04.135344 26798 heartbeater.cc:644] Failed to heartbeat to 127.9.123.126:45713 (0 consecutive failures): Network error: Failed to ping master at 127.9.123.126:45713: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
I20260926 15:07:04.136071 26800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:04.135786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.67:47463)
0926 15:07:04.135906 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:04.135970 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260926 15:07:04.136847 26800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:04.136541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.67:55673)
0926 15:07:04.136668 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:04.136745 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260926 15:07:04.137059 26798 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260926 15:07:04.139788 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 26670
I20260926 15:07:04.259615 26802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:04.258756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.65:35571)
0926 15:07:04.259255 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:04.259345 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":250,"threads_started":1}
I20260926 15:07:04.692502 26803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:04.691219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.66:45577)
0926 15:07:04.692050 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:04.692163 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":702,"thread_start_us":512,"threads_started":1}
I20260926 15:07:05.104172 26804 raft_consensus.cc:482] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:07:05.104499 26804 raft_consensus.cc:504] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } } peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } }
I20260926 15:07:05.106153 26804 leader_election.cc:288] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e823b55004f64499978fb94e96d2f89f (127.9.123.66:42499), 2566a89ae54442eca72b9b8add797384 (127.9.123.67:37657)
Sep 26 15:07:05 dist-test-slave-7153 krb5kdc[25215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435223, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for kudu/127.9.123.67@KRBTEST.COM
Sep 26 15:07:05 dist-test-slave-7153 krb5kdc[25215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435223, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for kudu/127.9.123.66@KRBTEST.COM
I20260926 15:07:05.135851 26805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:05.106465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:42499 (local address 127.9.123.65:37039)
0926 15:07:05.107073 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:05.107112 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:05.107710 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:05.111599 (+ 3889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:05.111611 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:05.112506 (+ 895us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:07:05.113317 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:05.113340 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:05.114927 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:05.114930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:05.116380 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:05.116394 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:05.117466 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:05.117473 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:05.117757 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:05.124274 (+ 6517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:07:05.124314 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:07:05.127157 (+ 2843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:07:05.132321 (+ 5164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:07:05.132326 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:07:05.132330 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:07:05.132859 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:07:05.133436 (+ 577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:07:05.133440 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:07:05.133443 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:07:05.133618 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:07:05.134576 (+ 958us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:07:05.134582 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:07:05.134881 (+ 299us) client_negotiation.cc:769] Sending connection context
0926 15:07:05.135106 (+ 225us) client_negotiation.cc:240] Negotiation successful
0926 15:07:05.135525 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":153,"spinlock_wait_cycles":32128,"thread_start_us":301,"threads_started":1}
I20260926 15:07:05.136065 26806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:05.106756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:37039 (local address 127.9.123.66:42499)
0926 15:07:05.108819 (+ 2063us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:05.108835 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:05.108880 (+ 45us) server_negotiation.cc:407] Connection header received
0926 15:07:05.110607 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:05.110617 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:05.111067 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:05.111406 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:05.113527 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:05.114728 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:05.116537 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:05.117226 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:05.117389 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:05.127491 (+ 10102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:05.127501 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:05.127514 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:05.127570 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:05.132167 (+ 4597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:05.133017 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:05.133025 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:05.133033 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:05.133299 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:05.133753 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:05.133757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:05.133761 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:05.134429 (+ 668us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:05.134611 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:05.135366 (+ 755us) server_negotiation.cc:299] Negotiation successful
0926 15:07:05.135745 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1878,"thread_start_us":1717,"threads_started":1}
I20260926 15:07:05.136858 26807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:05.107213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:37657 (local address 127.9.123.65:51839)
0926 15:07:05.108480 (+ 1267us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:05.108499 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:05.108603 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:05.111600 (+ 2997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:05.111611 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:05.112507 (+ 896us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:07:05.113318 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:05.113340 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:05.114925 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:05.114928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:05.116380 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:05.116394 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:05.117458 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:05.117466 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:05.117762 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:05.124273 (+ 6511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:07:05.124312 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:07:05.126970 (+ 2658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:07:05.132029 (+ 5059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:07:05.132039 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:07:05.132058 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:07:05.132671 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:07:05.133269 (+ 598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:07:05.133275 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:07:05.133278 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:07:05.133549 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:07:05.136080 (+ 2531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:07:05.136085 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:07:05.136229 (+ 144us) client_negotiation.cc:769] Sending connection context
0926 15:07:05.136360 (+ 131us) client_negotiation.cc:240] Negotiation successful
0926 15:07:05.136565 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1149,"mutex_wait_us":172,"spinlock_wait_cycles":19712,"thread_start_us":995,"threads_started":1}
I20260926 15:07:05.137761 26808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:05.107562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:51839 (local address 127.9.123.67:37657)
0926 15:07:05.109064 (+ 1502us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:05.109077 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:05.109138 (+ 61us) server_negotiation.cc:407] Connection header received
0926 15:07:05.110678 (+ 1540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:05.110688 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:05.111067 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:05.111414 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:05.113526 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:05.114727 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:05.116540 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:05.117227 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:05.117390 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:05.127448 (+ 10058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:05.127464 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:05.127489 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:05.127571 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:05.131789 (+ 4218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:05.132821 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:05.132832 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:05.132841 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:05.133082 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:05.133664 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:05.133670 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:05.133673 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:05.134429 (+ 756us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:05.134621 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:05.136672 (+ 2051us) server_negotiation.cc:299] Negotiation successful
0926 15:07:05.137342 (+ 670us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1349,"thread_start_us":1251,"threads_started":1}
I20260926 15:07:05.138583 26809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:05.137615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.67:56933)
0926 15:07:05.138113 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:05.138298 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":390,"threads_started":1}
W20260926 15:07:05.139664 26421 leader_election.cc:341] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e823b55004f64499978fb94e96d2f89f (127.9.123.66:42499): Not found: Tablet not found: 68da91940810430790b78a8f0a2b6730
W20260926 15:07:05.140556 26421 leader_election.cc:341] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2566a89ae54442eca72b9b8add797384 (127.9.123.67:37657): Not found: Tablet not found: 68da91940810430790b78a8f0a2b6730
I20260926 15:07:05.140700 26421 leader_election.cc:302] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bd7598679d84b62bbb2a0082717b051; no voters: 2566a89ae54442eca72b9b8add797384, e823b55004f64499978fb94e96d2f89f
I20260926 15:07:05.141084 26804 raft_consensus.cc:2726] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 15:07:05.260995 26807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:05.260569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.65:39143)
0926 15:07:05.260753 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:05.260845 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260926 15:07:05.694612 26810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:05.693825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.66:49051)
0926 15:07:05.694348 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:05.694443 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":268,"threads_started":1}
I20260926 15:07:06.140655 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:06.139924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.67:57987)
0926 15:07:06.140409 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:06.140494 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":262,"threads_started":1}
I20260926 15:07:06.262759 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:06.261983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.65:46143)
0926 15:07:06.262491 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:06.262575 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":284,"threads_started":1}
I20260926 15:07:06.696254 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:06.695538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.66:45483)
0926 15:07:06.695985 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:06.696086 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":231,"threads_started":1}
I20260926 15:07:06.981503 26814 raft_consensus.cc:482] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:07:06.981690 26814 raft_consensus.cc:504] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } } peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } }
I20260926 15:07:06.982331 26814 leader_election.cc:288] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e823b55004f64499978fb94e96d2f89f (127.9.123.66:42499), 2566a89ae54442eca72b9b8add797384 (127.9.123.67:37657)
W20260926 15:07:06.983444 26421 leader_election.cc:341] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2566a89ae54442eca72b9b8add797384 (127.9.123.67:37657): Not found: Tablet not found: 68da91940810430790b78a8f0a2b6730
W20260926 15:07:06.983719 26421 leader_election.cc:341] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e823b55004f64499978fb94e96d2f89f (127.9.123.66:42499): Not found: Tablet not found: 68da91940810430790b78a8f0a2b6730
I20260926 15:07:06.983817 26421 leader_election.cc:302] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bd7598679d84b62bbb2a0082717b051; no voters: 2566a89ae54442eca72b9b8add797384, e823b55004f64499978fb94e96d2f89f
I20260926 15:07:06.984130 26814 raft_consensus.cc:2726] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 15:07:07.142185 26815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:07.141509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.67:34423)
0926 15:07:07.141946 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:07.142031 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":233,"threads_started":1}
I20260926 15:07:07.264822 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:07.263858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.65:35129)
0926 15:07:07.264426 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:07.264584 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":280,"threads_started":1}
I20260926 15:07:07.698033 26817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:07.697246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.66:47383)
0926 15:07:07.697749 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:07.697844 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":259,"threads_started":1}
I20260926 15:07:08.143713 26818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:08.143039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.67:39315)
0926 15:07:08.143455 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:08.143533 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":215,"threads_started":1}
I20260926 15:07:08.266930 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:08.266143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.65:37593)
0926 15:07:08.266651 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:08.266741 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":253,"threads_started":1}
I20260926 15:07:08.699783 26820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:08.699051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.66:46775)
0926 15:07:08.699524 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:08.699614 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":248,"threads_started":1}
I20260926 15:07:09.145500 26821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:09.144631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.67:37195)
0926 15:07:09.145193 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:09.145291 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":286,"threads_started":1}
I20260926 15:07:09.268489 26822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:09.267749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.65:43707)
0926 15:07:09.268211 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:09.268304 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":250,"threads_started":1}
I20260926 15:07:09.701683 26823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:09.700860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.66:56295)
0926 15:07:09.701402 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:09.701498 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":298,"threads_started":1}
I20260926 15:07:10.147256 26824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:10.146508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.67:35697)
0926 15:07:10.146999 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:10.147087 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":250,"threads_started":1}
I20260926 15:07:10.270257 26825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:10.269506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.65:53247)
0926 15:07:10.269953 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:10.270042 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":213,"threads_started":1}
I20260926 15:07:10.358587 26826 raft_consensus.cc:482] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:07:10.358796 26826 raft_consensus.cc:504] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } } peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } }
I20260926 15:07:10.359527 26826 leader_election.cc:288] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e823b55004f64499978fb94e96d2f89f (127.9.123.66:42499), 2566a89ae54442eca72b9b8add797384 (127.9.123.67:37657)
W20260926 15:07:10.360533 26421 leader_election.cc:341] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e823b55004f64499978fb94e96d2f89f (127.9.123.66:42499): Not found: Tablet not found: 68da91940810430790b78a8f0a2b6730
W20260926 15:07:10.360734 26421 leader_election.cc:341] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2566a89ae54442eca72b9b8add797384 (127.9.123.67:37657): Not found: Tablet not found: 68da91940810430790b78a8f0a2b6730
I20260926 15:07:10.360828 26421 leader_election.cc:302] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bd7598679d84b62bbb2a0082717b051; no voters: 2566a89ae54442eca72b9b8add797384, e823b55004f64499978fb94e96d2f89f
I20260926 15:07:10.361090 26826 raft_consensus.cc:2726] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 15:07:10.703653 26827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:10.702712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.66:57537)
0926 15:07:10.703350 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:10.703458 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":322,"threads_started":1}
I20260926 15:07:11.149155 26828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:11.148206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.67:36921)
0926 15:07:11.148764 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:11.148931 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":290,"threads_started":1}
I20260926 15:07:11.272049 26829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:11.271316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.65:54061)
0926 15:07:11.271764 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:11.271887 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":231,"threads_started":1}
I20260926 15:07:11.705587 26830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:11.704715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.66:34607)
0926 15:07:11.705257 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:11.705372 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":275,"threads_started":1}
I20260926 15:07:12.150889 26831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:12.150084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.67:58693)
0926 15:07:12.150576 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:12.150694 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":256,"threads_started":1}
I20260926 15:07:12.273875 26832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:12.273042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.65:54005)
0926 15:07:12.273575 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:12.273678 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":302,"threads_started":1}
I20260926 15:07:12.458613 26833 raft_consensus.cc:482] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:07:12.458842 26833 raft_consensus.cc:504] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e823b55004f64499978fb94e96d2f89f" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42499 } } peers { permanent_uuid: "2566a89ae54442eca72b9b8add797384" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 37657 } } peers { permanent_uuid: "3bd7598679d84b62bbb2a0082717b051" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45717 } }
I20260926 15:07:12.459446 26833 leader_election.cc:288] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e823b55004f64499978fb94e96d2f89f (127.9.123.66:42499), 2566a89ae54442eca72b9b8add797384 (127.9.123.67:37657)
W20260926 15:07:12.460386 26421 leader_election.cc:341] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e823b55004f64499978fb94e96d2f89f (127.9.123.66:42499): Not found: Tablet not found: 68da91940810430790b78a8f0a2b6730
W20260926 15:07:12.460597 26421 leader_election.cc:341] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2566a89ae54442eca72b9b8add797384 (127.9.123.67:37657): Not found: Tablet not found: 68da91940810430790b78a8f0a2b6730
I20260926 15:07:12.460685 26421 leader_election.cc:302] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bd7598679d84b62bbb2a0082717b051; no voters: 2566a89ae54442eca72b9b8add797384, e823b55004f64499978fb94e96d2f89f
I20260926 15:07:12.460953 26833 raft_consensus.cc:2726] T 68da91940810430790b78a8f0a2b6730 P 3bd7598679d84b62bbb2a0082717b051 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260926 15:07:12.707367 26834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:12.706573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.66:37037)
0926 15:07:12.707100 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:12.707200 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":253,"threads_started":1}
I20260926 15:07:13.152613 26835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:13.151845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.67:49865)
0926 15:07:13.152353 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:13.152445 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":247,"threads_started":1}
I20260926 15:07:13.275643 26836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:13.274852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.65:41187)
0926 15:07:13.275378 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:13.275467 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":286,"threads_started":1}
I20260926 15:07:13.709306 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:13.708435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.66:41095)
0926 15:07:13.708968 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:13.709057 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":291,"threads_started":1}
I20260926 15:07:14.154382 26838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:14.153602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.67:36087)
0926 15:07:14.154079 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:14.154173 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":252,"threads_started":1}
I20260926 15:07:14.277474 26839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:14.276667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.65:35137)
0926 15:07:14.277207 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:14.277294 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":265,"threads_started":1}
I20260926 15:07:14.711712 26840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:14.710724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.66:42993)
0926 15:07:14.711391 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:14.711500 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":313,"threads_started":1}
I20260926 15:07:15.156231 26841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:15.155469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.67:52449)
0926 15:07:15.155926 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:15.156032 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":220,"threads_started":1}
I20260926 15:07:15.279258 26842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:15.278439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.65:49337)
0926 15:07:15.278969 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:15.279073 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":263,"threads_started":1}
I20260926 15:07:15.713472 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:15.712714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.66:51961)
0926 15:07:15.713204 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:15.713300 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":220,"threads_started":1}
I20260926 15:07:16.157934 26844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:16.157219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.67:48783)
0926 15:07:16.157673 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:16.157758 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":220,"threads_started":1}
I20260926 15:07:16.281153 26845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:16.280226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.65:54975)
0926 15:07:16.280807 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:16.280903 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":303,"threads_started":1}
I20260926 15:07:16.715209 26846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:16.714447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.66:41611)
0926 15:07:16.714948 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:16.715051 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":247,"threads_started":1}
I20260926 15:07:17.159699 26847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:17.158855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.67:59535)
0926 15:07:17.159397 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:17.159530 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":238,"threads_started":1}
I20260926 15:07:17.282881 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:17.282083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.65:43787)
0926 15:07:17.282597 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:17.282680 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":284,"threads_started":1}
I20260926 15:07:17.717017 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:17.716233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.66:33013)
0926 15:07:17.716723 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:17.716814 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":271,"threads_started":1}
I20260926 15:07:18.161346 26850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:18.160575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.67:41671)
0926 15:07:18.161017 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:18.161129 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":211,"threads_started":1}
I20260926 15:07:18.284838 26851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:18.284031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.65:58103)
0926 15:07:18.284559 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:18.284646 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":268,"threads_started":1}
I20260926 15:07:18.718919 26852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:18.718214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.66:52175)
0926 15:07:18.718665 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:18.718758 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":222,"threads_started":1}
I20260926 15:07:19.143772 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 26365
I20260926 15:07:19.163322 26854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:19.162472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45713 (local address 127.9.123.67:46237)
0926 15:07:19.162972 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:19.163116 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.126:45713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":218,"threads_started":1}
I20260926 15:07:19.165266 26853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:19.144855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37104 (local address 127.9.123.65:45717)
0926 15:07:19.145441 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:19.145452 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:19.145527 (+ 75us) server_negotiation.cc:407] Connection header received
0926 15:07:19.145680 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:19.145688 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:19.145931 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:19.146080 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:19.147379 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:19.148370 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:19.149818 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:19.150445 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:19.150556 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:19.159060 (+ 8504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:19.159077 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:19.159093 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:19.159135 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:19.161975 (+ 2840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:19.162974 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:19.162985 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:19.162989 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:19.163178 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:19.163618 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:19.163622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:19.163625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:19.164026 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:19.164294 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:19.164645 (+ 351us) server_negotiation.cc:299] Negotiation successful
0926 15:07:19.164904 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":274,"threads_started":1}
I20260926 15:07:19.166669 26530 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:19.258065 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 26365
I20260926 15:07:19.272742 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 26538
I20260926 15:07:19.286082 26856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:19.273553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51642 (local address 127.9.123.66:42499)
0926 15:07:19.273982 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:19.273987 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:19.274003 (+ 16us) server_negotiation.cc:407] Connection header received
0926 15:07:19.274061 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:19.274065 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:19.274128 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:19.274220 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:19.275438 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:19.276186 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:19.277644 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:19.278170 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:19.278283 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:19.280879 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:19.280884 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:19.280888 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:19.280931 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:19.283532 (+ 2601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:19.284216 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:19.284221 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:19.284223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:19.284377 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:19.284680 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:19.284683 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:19.284686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:19.284998 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:19.285272 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:19.285583 (+ 311us) server_negotiation.cc:299] Negotiation successful
0926 15:07:19.285769 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":223,"threads_started":1}
I20260926 15:07:19.286767 26662 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:19.371024 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 26538
I20260926 15:07:19.385530 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 26670
I20260926 15:07:19.408124 26858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:19.386375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36436 (local address 127.9.123.67:37657)
0926 15:07:19.386890 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:19.386895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:19.386912 (+ 17us) server_negotiation.cc:407] Connection header received
0926 15:07:19.386967 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:19.386971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:19.387030 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:19.387131 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:19.388542 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:19.389419 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:19.390675 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:19.391119 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:19.391229 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:19.402815 (+ 11586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:19.402820 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:19.402823 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:19.402868 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:19.405450 (+ 2582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:19.406175 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:19.406180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:19.406182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:19.406320 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:19.406709 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:19.406713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:19.406715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:19.407004 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:19.407263 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:19.407542 (+ 279us) server_negotiation.cc:299] Negotiation successful
0926 15:07:19.407759 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":264,"threads_started":1}
I20260926 15:07:19.408859 26793 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:19.505362 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 26670
2026-09-26T15:07:19Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64146 ms)
[----------] 30 tests from SecurityITest (300373 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:07:19 dist-test-slave-7153 krb5kdc[26862](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:07:19 dist-test-slave-7153 krb5kdc[26862](info): set up 2 sockets
Sep 26 15:07:19 dist-test-slave-7153 krb5kdc[26862](info): commencing operation
krb5kdc: starting...
W20260926 15:07:20.541085 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.007s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:07:20 dist-test-slave-7153 krb5kdc[26862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:07:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:07:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:07:20.777467 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:39593
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:34137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:39593
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:07:21.112978 26875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:21.113337 26875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:21.113487 26875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:21.120795 26875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:07:21.120919 26875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:21.121001 26875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:21.121073 26875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:07:21.121157 26875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:07:21.131513 26875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:34137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:39593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:39593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:21.132872 26875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:21.134239 26875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:21.146144 26880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:21.146375 26875 server_base.cc:1034] running on GCE node
W20260926 15:07:21.146112 26883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:21.146203 26881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:21.147385 26875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:21.148803 26875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:21.149998 26875 hybrid_clock.cc:648] HybridClock initialized: now 1790435241149927 us; error 62 us; skew 500 ppm
I20260926 15:07:21.150357 26875 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:21 dist-test-slave-7153 krb5kdc[26862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435241, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:21.153561 26875 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:07:21.155120 26875 webserver.cc:460] Webserver started at http://127.9.123.126:42991/ using document root <none> and password file <none>
I20260926 15:07:21.155830 26875 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:21.155948 26875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:21.156260 26875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:21.159504 26875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "859b69e557e34c31850a5752792620bb"
format_stamp: "Formatted at 2026-09-26 15:07:21 on dist-test-slave-7153"
server_key: "23a3fa4ccf6545d5c4817bc88a236684"
server_key_iv: "a712f3f20413024a08e09ac0ea364421"
server_key_version: "encryptionkey@0"
I20260926 15:07:21.160280 26875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "859b69e557e34c31850a5752792620bb"
format_stamp: "Formatted at 2026-09-26 15:07:21 on dist-test-slave-7153"
server_key: "23a3fa4ccf6545d5c4817bc88a236684"
server_key_iv: "a712f3f20413024a08e09ac0ea364421"
server_key_version: "encryptionkey@0"
I20260926 15:07:21.165719 26875 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260926 15:07:21.169422 26890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:21.170835 26875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:07:21.170987 26875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "859b69e557e34c31850a5752792620bb"
format_stamp: "Formatted at 2026-09-26 15:07:21 on dist-test-slave-7153"
server_key: "23a3fa4ccf6545d5c4817bc88a236684"
server_key_iv: "a712f3f20413024a08e09ac0ea364421"
server_key_version: "encryptionkey@0"
I20260926 15:07:21.171177 26875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:21.183921 26875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:21.187891 26875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:21.188153 26875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:21.205981 26875 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:39593
I20260926 15:07:21.206003 26941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:39593 every 8 connection(s)
I20260926 15:07:21.207469 26875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:07:21.211367 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 26875
I20260926 15:07:21.211583 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:07:21.211608 26942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:21.212457 9709 external_mini_cluster.cc:1428] Setting key 0989d066e54f6fffeeab51e2a0094cae
I20260926 15:07:21.226176 26942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 859b69e557e34c31850a5752792620bb: Bootstrap starting.
Sep 26 15:07:21 dist-test-slave-7153 krb5kdc[26862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:07:21.230496 26942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 859b69e557e34c31850a5752792620bb: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:21.231738 26942 log.cc:824] T 00000000000000000000000000000000 P 859b69e557e34c31850a5752792620bb: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:21.235625 26942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 859b69e557e34c31850a5752792620bb: No bootstrap required, opened a new log
I20260926 15:07:21.239627 26945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:21.214693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35202 (local address 127.9.123.126:39593)
0926 15:07:21.215502 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:21.215514 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:21.215556 (+ 42us) server_negotiation.cc:407] Connection header received
0926 15:07:21.218038 (+ 2482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:21.218068 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:21.218605 (+ 537us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:21.218990 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:21.220895 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:21.222395 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:21.224266 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:21.225012 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:21.225237 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:21.230312 (+ 5075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:21.230324 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:21.230341 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:21.230388 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:21.234149 (+ 3761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:21.235194 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:21.235201 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:21.235208 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:21.235403 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:21.235896 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:21.235899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:21.235902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:21.236543 (+ 641us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:21.236788 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:21.237502 (+ 714us) server_negotiation.cc:299] Negotiation successful
0926 15:07:21.238150 (+ 648us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":374,"threads_started":1}
I20260926 15:07:21.244874 26942 raft_consensus.cc:348] T 00000000000000000000000000000000 P 859b69e557e34c31850a5752792620bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "859b69e557e34c31850a5752792620bb" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39593 } }
I20260926 15:07:21.245419 26942 raft_consensus.cc:374] T 00000000000000000000000000000000 P 859b69e557e34c31850a5752792620bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:21.245538 26942 raft_consensus.cc:729] T 00000000000000000000000000000000 P 859b69e557e34c31850a5752792620bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 859b69e557e34c31850a5752792620bb, State: Initialized, Role: FOLLOWER
I20260926 15:07:21.246181 26942 consensus_queue.cc:260] T 00000000000000000000000000000000 P 859b69e557e34c31850a5752792620bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "859b69e557e34c31850a5752792620bb" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39593 } }
I20260926 15:07:21.246537 26942 raft_consensus.cc:388] T 00000000000000000000000000000000 P 859b69e557e34c31850a5752792620bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:07:21.246690 26942 raft_consensus.cc:482] T 00000000000000000000000000000000 P 859b69e557e34c31850a5752792620bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:07:21.246858 26942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 859b69e557e34c31850a5752792620bb [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:21.249414 26942 raft_consensus.cc:504] T 00000000000000000000000000000000 P 859b69e557e34c31850a5752792620bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "859b69e557e34c31850a5752792620bb" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39593 } }
I20260926 15:07:21.249967 26942 leader_election.cc:302] T 00000000000000000000000000000000 P 859b69e557e34c31850a5752792620bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 859b69e557e34c31850a5752792620bb; no voters:
I20260926 15:07:21.250757 26942 leader_election.cc:288] T 00000000000000000000000000000000 P 859b69e557e34c31850a5752792620bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:07:21.250939 26947 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 859b69e557e34c31850a5752792620bb [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:07:21.252267 26947 raft_consensus.cc:686] T 00000000000000000000000000000000 P 859b69e557e34c31850a5752792620bb [term 1 LEADER]: Becoming Leader. State: Replica: 859b69e557e34c31850a5752792620bb, State: Running, Role: LEADER
I20260926 15:07:21.252821 26947 consensus_queue.cc:237] T 00000000000000000000000000000000 P 859b69e557e34c31850a5752792620bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "859b69e557e34c31850a5752792620bb" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39593 } }
I20260926 15:07:21.253425 26942 sys_catalog.cc:564] T 00000000000000000000000000000000 P 859b69e557e34c31850a5752792620bb [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:07:21.255544 26949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 859b69e557e34c31850a5752792620bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 859b69e557e34c31850a5752792620bb. Latest consensus state: current_term: 1 leader_uuid: "859b69e557e34c31850a5752792620bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "859b69e557e34c31850a5752792620bb" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39593 } } }
I20260926 15:07:21.255985 26949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 859b69e557e34c31850a5752792620bb [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:21.256567 26948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 859b69e557e34c31850a5752792620bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "859b69e557e34c31850a5752792620bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "859b69e557e34c31850a5752792620bb" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39593 } } }
I20260926 15:07:21.256901 26948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 859b69e557e34c31850a5752792620bb [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:21.261669 26954 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:07:21.267282 26954 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:07:21.276510 26954 catalog_manager.cc:1347] Generated new cluster ID: 6c583c78acd2465682dbea519137860b
I20260926 15:07:21.276638 26954 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:07:21.285368 26954 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:07:21.286639 26954 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:07:21.293928 26954 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 859b69e557e34c31850a5752792620bb: Generated new TSK 0
I20260926 15:07:21.294744 26954 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.123.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
I20260926 15:07:21.364410 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:39593
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:34137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:07:21.706640 26969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:21.706965 26969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:21.707094 26969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:21.714829 26969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:21.714969 26969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:21.715128 26969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:07:21.725416 26969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:34137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:39593
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:21.726778 26969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:21.728190 26969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:21.741525 26974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:21.741868 26977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:21.741894 26975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:21.742487 26969 server_base.cc:1034] running on GCE node
I20260926 15:07:21.743103 26969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:21.744068 26969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:21.745289 26969 hybrid_clock.cc:648] HybridClock initialized: now 1790435241745230 us; error 48 us; skew 500 ppm
I20260926 15:07:21.745584 26969 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:21 dist-test-slave-7153 krb5kdc[26862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435241, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:21.748651 26969 init.cc:377] Logged in from keytab as kudu/127.9.123.65@KRBTEST.COM (short username kudu)
I20260926 15:07:21.750150 26969 webserver.cc:460] Webserver started at http://127.9.123.65:37995/ using document root <none> and password file <none>
I20260926 15:07:21.750900 26969 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:21.751008 26969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:21.751320 26969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:21.754395 26969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "f29e38b721c7404f8b859746b4133c2f"
format_stamp: "Formatted at 2026-09-26 15:07:21 on dist-test-slave-7153"
server_key: "eb231f0fa65277dd1e992b5ee1f4d95b"
server_key_iv: "fc656447822c25e11919bcfef23cb6c3"
server_key_version: "encryptionkey@0"
I20260926 15:07:21.755168 26969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "f29e38b721c7404f8b859746b4133c2f"
format_stamp: "Formatted at 2026-09-26 15:07:21 on dist-test-slave-7153"
server_key: "eb231f0fa65277dd1e992b5ee1f4d95b"
server_key_iv: "fc656447822c25e11919bcfef23cb6c3"
server_key_version: "encryptionkey@0"
I20260926 15:07:21.760638 26969 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 15:07:21.764616 26984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:21.766078 26969 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:07:21.766248 26969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "f29e38b721c7404f8b859746b4133c2f"
format_stamp: "Formatted at 2026-09-26 15:07:21 on dist-test-slave-7153"
server_key: "eb231f0fa65277dd1e992b5ee1f4d95b"
server_key_iv: "fc656447822c25e11919bcfef23cb6c3"
server_key_version: "encryptionkey@0"
I20260926 15:07:21.766451 26969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:21.786764 26969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:21.791296 26969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:21.791605 26969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:21.792632 26969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:07:21.794185 26969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:07:21.794260 26969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:21.794380 26969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:07:21.794431 26969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:21.825552 26969 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:35371
I20260926 15:07:21.825582 27096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:35371 every 8 connection(s)
I20260926 15:07:21.826931 26969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:07:21.832590 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 26969
I20260926 15:07:21.832808 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:07:21.833518 9709 external_mini_cluster.cc:1428] Setting key c10935258c785df734b30174cbdef371
Sep 26 15:07:21 dist-test-slave-7153 krb5kdc[26862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435241, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:07:21.849782 27099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:21.829799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:60635 (local address 127.9.123.126:39593)
0926 15:07:21.830382 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:21.830389 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:21.831534 (+ 1145us) server_negotiation.cc:407] Connection header received
0926 15:07:21.833605 (+ 2071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:21.833623 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:21.833730 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:21.833883 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:21.835707 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:21.836781 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:21.838796 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:21.839367 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:21.839510 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:21.843340 (+ 3830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:21.843348 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:21.843353 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:21.843400 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:21.846074 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:21.847170 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:21.847175 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:21.847178 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:21.847344 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:21.847839 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:21.847842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:21.847844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:21.848185 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:21.848308 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:21.849304 (+ 996us) server_negotiation.cc:299] Negotiation successful
0926 15:07:21.849503 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":242,"threads_started":1}
I20260926 15:07:21.851166 27100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:21.830530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39593 (local address 127.9.123.65:60635)
0926 15:07:21.831346 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:21.831388 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:21.833273 (+ 1885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:21.834113 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:21.834123 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:21.834839 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:07:21.835498 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:21.835528 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:21.837012 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:21.837017 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:21.838609 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:21.838626 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:21.839631 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:21.839637 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:21.839875 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:21.840606 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:07:21.840636 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:07:21.843041 (+ 2405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:07:21.846262 (+ 3221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:07:21.846273 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:07:21.846295 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:07:21.846925 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:07:21.847459 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:07:21.847464 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:07:21.847468 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:07:21.847741 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:07:21.848311 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:07:21.848320 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:07:21.848785 (+ 465us) client_negotiation.cc:769] Sending connection context
0926 15:07:21.849128 (+ 343us) client_negotiation.cc:240] Negotiation successful
0926 15:07:21.849687 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":305,"threads_started":1}
I20260926 15:07:21.853009 27097 heartbeater.cc:347] Connected to a master server at 127.9.123.126:39593
I20260926 15:07:21.853413 27097 heartbeater.cc:464] Registering TS with master...
I20260926 15:07:21.854283 27097 heartbeater.cc:511] Master 127.9.123.126:39593 requested a full tablet report, sending...
I20260926 15:07:21.857046 26907 ts_manager.cc:194] Registered new tserver with Master: f29e38b721c7404f8b859746b4133c2f (127.9.123.65:35371)
I20260926 15:07:21.859429 26907 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.65@KRBTEST.COM'} at 127.9.123.65:60635
WARNING: no policy specified for kudu/127.9.123.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
I20260926 15:07:21.891642 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:39593
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:34137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:07:22.216252 27103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:22.216552 27103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:22.216693 27103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:22.224327 27103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:22.224490 27103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:22.224661 27103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:07:22.236127 27103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:34137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:39593
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:22.238127 27103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:22.239749 27103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:22.257437 27109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:22.257362 27108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:22.257388 27103 server_base.cc:1034] running on GCE node
W20260926 15:07:22.257848 27111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:22.258534 27103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:22.260035 27103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:22.261286 27103 hybrid_clock.cc:648] HybridClock initialized: now 1790435242261222 us; error 52 us; skew 500 ppm
I20260926 15:07:22.261653 27103 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:22 dist-test-slave-7153 krb5kdc[26862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435242, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:22.265205 27103 init.cc:377] Logged in from keytab as kudu/127.9.123.66@KRBTEST.COM (short username kudu)
I20260926 15:07:22.266868 27103 webserver.cc:460] Webserver started at http://127.9.123.66:40251/ using document root <none> and password file <none>
I20260926 15:07:22.267601 27103 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:22.267742 27103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:22.268054 27103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:22.271493 27103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "34758cb49b4f4778b4bfc6d97cd36000"
format_stamp: "Formatted at 2026-09-26 15:07:22 on dist-test-slave-7153"
server_key: "9be0af04327154701baadd2e8a415b81"
server_key_iv: "bf71dd29da3be990f2f3f763a186c372"
server_key_version: "encryptionkey@0"
I20260926 15:07:22.272357 27103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "34758cb49b4f4778b4bfc6d97cd36000"
format_stamp: "Formatted at 2026-09-26 15:07:22 on dist-test-slave-7153"
server_key: "9be0af04327154701baadd2e8a415b81"
server_key_iv: "bf71dd29da3be990f2f3f763a186c372"
server_key_version: "encryptionkey@0"
I20260926 15:07:22.278177 27103 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 15:07:22.282182 27118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:22.283872 27103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:07:22.284051 27103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "34758cb49b4f4778b4bfc6d97cd36000"
format_stamp: "Formatted at 2026-09-26 15:07:22 on dist-test-slave-7153"
server_key: "9be0af04327154701baadd2e8a415b81"
server_key_iv: "bf71dd29da3be990f2f3f763a186c372"
server_key_version: "encryptionkey@0"
I20260926 15:07:22.284267 27103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:22.331111 27103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:22.335457 27103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:22.335738 27103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:22.336753 27103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:07:22.338138 27103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:07:22.338215 27103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:22.338335 27103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:07:22.338392 27103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:22.370847 27103 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:44231
I20260926 15:07:22.370883 27230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:44231 every 8 connection(s)
I20260926 15:07:22.372260 27103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:07:22.374744 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 27103
I20260926 15:07:22.374933 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:07:22.375660 9709 external_mini_cluster.cc:1428] Setting key b1ca852e185b7e5a3180f704a06b71ab
Sep 26 15:07:22 dist-test-slave-7153 krb5kdc[26862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435242, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:07:22.395473 27233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:22.375096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:55559 (local address 127.9.123.126:39593)
0926 15:07:22.375620 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:22.375625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:22.377130 (+ 1505us) server_negotiation.cc:407] Connection header received
0926 15:07:22.379287 (+ 2157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:22.379293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:22.379382 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:22.379481 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:22.381192 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:22.382004 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:22.383690 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:22.384304 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:22.384457 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:22.388681 (+ 4224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:22.388690 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:22.388695 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:22.388751 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:22.391648 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:22.392732 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:22.392738 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:22.392740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:22.392907 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:22.393528 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:22.393532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:22.393534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:22.393864 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:22.394007 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:22.394979 (+ 972us) server_negotiation.cc:299] Negotiation successful
0926 15:07:22.395189 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":293,"threads_started":1}
I20260926 15:07:22.396780 27234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:22.375995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39593 (local address 127.9.123.66:55559)
0926 15:07:22.376928 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:22.376980 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:22.378987 (+ 2007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:22.379768 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:22.379777 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:22.380363 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:07:22.380977 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:22.380996 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:22.382185 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:22.382189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:22.383521 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:22.383535 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:22.384633 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:22.384642 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:22.384897 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:22.385641 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:07:22.385667 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:07:22.388262 (+ 2595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:07:22.391859 (+ 3597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:07:22.391871 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:07:22.391892 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:07:22.392555 (+ 663us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:07:22.393034 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:07:22.393038 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:07:22.393042 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:07:22.393374 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:07:22.394002 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:07:22.394012 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:07:22.394518 (+ 506us) client_negotiation.cc:769] Sending connection context
0926 15:07:22.394850 (+ 332us) client_negotiation.cc:240] Negotiation successful
0926 15:07:22.395465 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":395,"threads_started":1}
I20260926 15:07:22.398447 27231 heartbeater.cc:347] Connected to a master server at 127.9.123.126:39593
I20260926 15:07:22.398833 27231 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.9.123.67@KRBTEST.COM; defaulting to no policy
I20260926 15:07:22.399871 27231 heartbeater.cc:511] Master 127.9.123.126:39593 requested a full tablet report, sending...
I20260926 15:07:22.401816 26907 ts_manager.cc:194] Registered new tserver with Master: 34758cb49b4f4778b4bfc6d97cd36000 (127.9.123.66:44231)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.67@KRBTEST.COM" created.
I20260926 15:07:22.403090 26907 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.66@KRBTEST.COM'} at 127.9.123.66:55559
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
I20260926 15:07:22.433288 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:39593
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:34137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:07:22.767233 27237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:22.767535 27237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:22.767678 27237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:22.775517 27237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:22.775662 27237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:22.775851 27237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:07:22.785858 27237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:34137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:39593
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:22.787235 27237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:22.788656 27237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:22.801930 27243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:22.801975 27242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:22.801954 27245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:22.802441 27237 server_base.cc:1034] running on GCE node
I20260926 15:07:22.803252 27237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:22.804215 27237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:22.805403 27237 hybrid_clock.cc:648] HybridClock initialized: now 1790435242805341 us; error 55 us; skew 500 ppm
I20260926 15:07:22.805696 27237 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:22 dist-test-slave-7153 krb5kdc[26862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435242, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:22.808755 27237 init.cc:377] Logged in from keytab as kudu/127.9.123.67@KRBTEST.COM (short username kudu)
I20260926 15:07:22.810225 27237 webserver.cc:460] Webserver started at http://127.9.123.67:35995/ using document root <none> and password file <none>
I20260926 15:07:22.810902 27237 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:22.811014 27237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:22.811326 27237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:22.814355 27237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "8a2c6fb9f21f4c49a92b2a7531bb9f03"
format_stamp: "Formatted at 2026-09-26 15:07:22 on dist-test-slave-7153"
server_key: "478b3117b81432183ba7d6fa5c7ba365"
server_key_iv: "2492c78d8cb8b579e0c1b2141f789556"
server_key_version: "encryptionkey@0"
I20260926 15:07:22.815132 27237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "8a2c6fb9f21f4c49a92b2a7531bb9f03"
format_stamp: "Formatted at 2026-09-26 15:07:22 on dist-test-slave-7153"
server_key: "478b3117b81432183ba7d6fa5c7ba365"
server_key_iv: "2492c78d8cb8b579e0c1b2141f789556"
server_key_version: "encryptionkey@0"
I20260926 15:07:22.820520 27237 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260926 15:07:22.824280 27252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:22.825892 27237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:07:22.826045 27237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "8a2c6fb9f21f4c49a92b2a7531bb9f03"
format_stamp: "Formatted at 2026-09-26 15:07:22 on dist-test-slave-7153"
server_key: "478b3117b81432183ba7d6fa5c7ba365"
server_key_iv: "2492c78d8cb8b579e0c1b2141f789556"
server_key_version: "encryptionkey@0"
I20260926 15:07:22.826252 27237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:22.849581 27237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:22.854030 27237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:22.854314 27237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:22.855372 27237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:07:22.856802 27237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:07:22.856882 27237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:22.856981 27237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:07:22.857034 27237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:22.863696 27097 heartbeater.cc:503] Master 127.9.123.126:39593 was elected leader, sending a full tablet report...
I20260926 15:07:22.887949 27237 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:35807
I20260926 15:07:22.888015 27364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:35807 every 8 connection(s)
I20260926 15:07:22.889348 27237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:07:22.896862 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 27237
I20260926 15:07:22.897010 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:07:22.897481 9709 external_mini_cluster.cc:1428] Setting key 6da11b3d923e1832118dfcd07651894f
Sep 26 15:07:22 dist-test-slave-7153 krb5kdc[26862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435242, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:07:22.908965 27233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:22.891901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:48819 (local address 127.9.123.126:39593)
0926 15:07:22.892096 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:22.892102 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:22.893323 (+ 1221us) server_negotiation.cc:407] Connection header received
0926 15:07:22.895180 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:22.895184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:22.895262 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:22.895352 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:22.897073 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:22.897926 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:22.899641 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:22.900231 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:22.900351 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:22.903877 (+ 3526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:22.903883 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:22.903887 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:22.903921 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:22.906259 (+ 2338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:22.906993 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:22.906997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:22.906999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:22.907123 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:22.907475 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:22.907477 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:22.907480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:22.907741 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:22.907871 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:22.908551 (+ 680us) server_negotiation.cc:299] Negotiation successful
0926 15:07:22.908734 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 15:07:22.910060 27367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:22.892425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39593 (local address 127.9.123.67:48819)
0926 15:07:22.893152 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:22.893207 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:22.894905 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:22.895623 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:22.895631 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:22.896222 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:07:22.896834 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:22.896850 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:22.898090 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:22.898097 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:22.899520 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:22.899533 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:22.900433 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:22.900440 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:22.900685 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:22.901420 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:07:22.901443 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:07:22.903659 (+ 2216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:07:22.906388 (+ 2729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:07:22.906395 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:07:22.906410 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:07:22.906879 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:07:22.907203 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:07:22.907206 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:07:22.907209 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:07:22.907393 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:07:22.907854 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:07:22.907860 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:07:22.908217 (+ 357us) client_negotiation.cc:769] Sending connection context
0926 15:07:22.908453 (+ 236us) client_negotiation.cc:240] Negotiation successful
0926 15:07:22.908893 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":260,"threads_started":1}
I20260926 15:07:22.911549 27365 heartbeater.cc:347] Connected to a master server at 127.9.123.126:39593
I20260926 15:07:22.911859 27365 heartbeater.cc:464] Registering TS with master...
I20260926 15:07:22.912565 27365 heartbeater.cc:511] Master 127.9.123.126:39593 requested a full tablet report, sending...
I20260926 15:07:22.913988 26907 ts_manager.cc:194] Registered new tserver with Master: 8a2c6fb9f21f4c49a92b2a7531bb9f03 (127.9.123.67:35807)
I20260926 15:07:22.914844 26907 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.67@KRBTEST.COM'} at 127.9.123.67:48819
I20260926 15:07:22.925432 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 15:07:22.945310 27233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:22.930575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35212 (local address 127.9.123.126:39593)
0926 15:07:22.930811 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:22.930818 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:22.931434 (+ 616us) server_negotiation.cc:407] Connection header received
0926 15:07:22.931568 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:22.931572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:22.931661 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:22.931756 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:22.933390 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:22.934242 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:22.935691 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:22.936168 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:22.936286 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:22.939122 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:22.939128 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:22.939131 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:22.939174 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:22.942007 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:22.942939 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:22.942944 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:22.942947 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:22.943109 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:22.943582 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:22.943591 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:22.943595 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:22.944094 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:22.944322 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:22.944704 (+ 382us) server_negotiation.cc:299] Negotiation successful
0926 15:07:22.944904 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 15:07:22.949849 26907 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:07:22.957077 26907 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:07:22.982473 27377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:22.970788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:35371 (local address 127.0.0.1:52010)
0926 15:07:22.971735 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:22.971792 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:22.971977 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:22.973370 (+ 1393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:22.973380 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:22.973419 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:22.973900 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:22.973914 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:22.976446 (+ 2532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:22.976455 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:22.978942 (+ 2487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:22.978965 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:22.981328 (+ 2363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:22.981337 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:22.981646 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:22.981708 (+ 62us) client_negotiation.cc:769] Sending connection context
0926 15:07:22.981828 (+ 120us) client_negotiation.cc:240] Negotiation successful
0926 15:07:22.982066 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":519,"threads_started":1}
I20260926 15:07:22.984686 27380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:22.971952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52010 (local address 127.9.123.65:35371)
0926 15:07:22.972588 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:22.972598 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:22.972629 (+ 31us) server_negotiation.cc:407] Connection header received
0926 15:07:22.972739 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:22.972748 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:22.973046 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:22.974131 (+ 1085us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:22.974443 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:22.976216 (+ 1773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:22.979231 (+ 3015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:22.980983 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:22.983658 (+ 2675us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:22.983987 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:22.984079 (+ 92us) server_negotiation.cc:299] Negotiation successful
0926 15:07:22.984308 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":310,"threads_started":1}
I20260926 15:07:22.989521 27032 tablet_service.cc:1467] Processing CreateTablet for tablet a2239dc1e1ae4de78796b7270b4900c2 (DEFAULT_TABLE table=test-table [id=afe7eae7661d4a4ea65598e138ab3c41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:07:22.990833 27377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:22.972311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:44231 (local address 127.0.0.1:34788)
0926 15:07:22.982998 (+ 10687us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:22.983016 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:22.983117 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:22.984579 (+ 1462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:22.984585 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:22.984612 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:22.984807 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:22.984820 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:22.986831 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:22.986838 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:22.988530 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:22.988547 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:22.990080 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:22.990086 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:22.990283 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:22.990319 (+ 36us) client_negotiation.cc:769] Sending connection context
0926 15:07:22.990413 (+ 94us) client_negotiation.cc:240] Negotiation successful
0926 15:07:22.990557 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10584,"mutex_wait_us":30,"thread_start_us":9662,"threads_started":1}
I20260926 15:07:22.991324 27032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2239dc1e1ae4de78796b7270b4900c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:22.991571 27382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:22.972835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34788 (local address 127.9.123.66:44231)
0926 15:07:22.984054 (+ 11219us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:22.984063 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:22.984089 (+ 26us) server_negotiation.cc:407] Connection header received
0926 15:07:22.984177 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:22.984183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:22.984373 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:22.984559 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:22.984995 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:22.986648 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:22.988686 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:22.989871 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:22.990034 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:22.990259 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:22.991128 (+ 869us) server_negotiation.cc:299] Negotiation successful
0926 15:07:22.991290 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11076,"thread_start_us":10958,"threads_started":1}
I20260926 15:07:22.993086 27379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:22.971517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42464 (local address 127.9.123.67:35807)
0926 15:07:22.981562 (+ 10045us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:22.981572 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:22.982655 (+ 1083us) server_negotiation.cc:407] Connection header received
0926 15:07:22.982803 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:22.982813 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:22.983161 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:22.983391 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:22.985535 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:22.987067 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:22.990310 (+ 3243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:22.991794 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:22.991941 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:22.992173 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:22.992720 (+ 547us) server_negotiation.cc:299] Negotiation successful
0926 15:07:22.992859 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9800,"thread_start_us":10071,"threads_started":1}
I20260926 15:07:22.993147 27381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:22.971023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:35807 (local address 127.0.0.1:42464)
0926 15:07:22.982275 (+ 11252us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:22.982300 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:22.982473 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:22.984829 (+ 2356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:22.984833 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:22.984854 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:22.985294 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:22.985306 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:22.987258 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:22.987263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:22.990077 (+ 2814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:22.990095 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:22.992329 (+ 2234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:22.992336 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:22.992517 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:22.992551 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 15:07:22.992638 (+ 87us) client_negotiation.cc:240] Negotiation successful
0926 15:07:22.992788 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11100,"thread_start_us":16900,"threads_started":1}
I20260926 15:07:22.997128 27166 tablet_service.cc:1467] Processing CreateTablet for tablet a2239dc1e1ae4de78796b7270b4900c2 (DEFAULT_TABLE table=test-table [id=afe7eae7661d4a4ea65598e138ab3c41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:07:22.997130 27300 tablet_service.cc:1467] Processing CreateTablet for tablet a2239dc1e1ae4de78796b7270b4900c2 (DEFAULT_TABLE table=test-table [id=afe7eae7661d4a4ea65598e138ab3c41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:07:22.998802 27166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2239dc1e1ae4de78796b7270b4900c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:22.999356 27300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2239dc1e1ae4de78796b7270b4900c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:23.003973 27383 tablet_bootstrap.cc:492] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f: Bootstrap starting.
I20260926 15:07:23.007921 27383 tablet_bootstrap.cc:654] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:23.008996 27383 log.cc:824] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:23.011806 27383 tablet_bootstrap.cc:492] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f: No bootstrap required, opened a new log
I20260926 15:07:23.012104 27383 ts_tablet_manager.cc:1397] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260926 15:07:23.015154 27386 tablet_bootstrap.cc:492] T a2239dc1e1ae4de78796b7270b4900c2 P 8a2c6fb9f21f4c49a92b2a7531bb9f03: Bootstrap starting.
I20260926 15:07:23.016079 27385 tablet_bootstrap.cc:492] T a2239dc1e1ae4de78796b7270b4900c2 P 34758cb49b4f4778b4bfc6d97cd36000: Bootstrap starting.
I20260926 15:07:23.019923 27385 tablet_bootstrap.cc:654] T a2239dc1e1ae4de78796b7270b4900c2 P 34758cb49b4f4778b4bfc6d97cd36000: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:23.020002 27386 tablet_bootstrap.cc:654] T a2239dc1e1ae4de78796b7270b4900c2 P 8a2c6fb9f21f4c49a92b2a7531bb9f03: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:23.021063 27385 log.cc:824] T a2239dc1e1ae4de78796b7270b4900c2 P 34758cb49b4f4778b4bfc6d97cd36000: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:23.021294 27386 log.cc:824] T a2239dc1e1ae4de78796b7270b4900c2 P 8a2c6fb9f21f4c49a92b2a7531bb9f03: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:23.022727 27383 raft_consensus.cc:348] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29e38b721c7404f8b859746b4133c2f" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 35371 } } peers { permanent_uuid: "8a2c6fb9f21f4c49a92b2a7531bb9f03" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35807 } } peers { permanent_uuid: "34758cb49b4f4778b4bfc6d97cd36000" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 44231 } }
I20260926 15:07:23.023139 27383 raft_consensus.cc:374] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:23.023257 27383 raft_consensus.cc:729] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f29e38b721c7404f8b859746b4133c2f, State: Initialized, Role: FOLLOWER
I20260926 15:07:23.023890 27383 consensus_queue.cc:260] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29e38b721c7404f8b859746b4133c2f" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 35371 } } peers { permanent_uuid: "8a2c6fb9f21f4c49a92b2a7531bb9f03" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35807 } } peers { permanent_uuid: "34758cb49b4f4778b4bfc6d97cd36000" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 44231 } }
I20260926 15:07:23.024639 27385 tablet_bootstrap.cc:492] T a2239dc1e1ae4de78796b7270b4900c2 P 34758cb49b4f4778b4bfc6d97cd36000: No bootstrap required, opened a new log
I20260926 15:07:23.025000 27385 ts_tablet_manager.cc:1397] T a2239dc1e1ae4de78796b7270b4900c2 P 34758cb49b4f4778b4bfc6d97cd36000: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.007s
I20260926 15:07:23.025719 27383 ts_tablet_manager.cc:1428] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260926 15:07:23.026271 27386 tablet_bootstrap.cc:492] T a2239dc1e1ae4de78796b7270b4900c2 P 8a2c6fb9f21f4c49a92b2a7531bb9f03: No bootstrap required, opened a new log
I20260926 15:07:23.026671 27386 ts_tablet_manager.cc:1397] T a2239dc1e1ae4de78796b7270b4900c2 P 8a2c6fb9f21f4c49a92b2a7531bb9f03: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260926 15:07:23.033986 27385 raft_consensus.cc:348] T a2239dc1e1ae4de78796b7270b4900c2 P 34758cb49b4f4778b4bfc6d97cd36000 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29e38b721c7404f8b859746b4133c2f" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 35371 } } peers { permanent_uuid: "8a2c6fb9f21f4c49a92b2a7531bb9f03" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35807 } } peers { permanent_uuid: "34758cb49b4f4778b4bfc6d97cd36000" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 44231 } }
I20260926 15:07:23.034417 27385 raft_consensus.cc:374] T a2239dc1e1ae4de78796b7270b4900c2 P 34758cb49b4f4778b4bfc6d97cd36000 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:23.034516 27385 raft_consensus.cc:729] T a2239dc1e1ae4de78796b7270b4900c2 P 34758cb49b4f4778b4bfc6d97cd36000 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34758cb49b4f4778b4bfc6d97cd36000, State: Initialized, Role: FOLLOWER
I20260926 15:07:23.035130 27385 consensus_queue.cc:260] T a2239dc1e1ae4de78796b7270b4900c2 P 34758cb49b4f4778b4bfc6d97cd36000 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29e38b721c7404f8b859746b4133c2f" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 35371 } } peers { permanent_uuid: "8a2c6fb9f21f4c49a92b2a7531bb9f03" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35807 } } peers { permanent_uuid: "34758cb49b4f4778b4bfc6d97cd36000" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 44231 } }
I20260926 15:07:23.036334 27231 heartbeater.cc:503] Master 127.9.123.126:39593 was elected leader, sending a full tablet report...
I20260926 15:07:23.036181 27386 raft_consensus.cc:348] T a2239dc1e1ae4de78796b7270b4900c2 P 8a2c6fb9f21f4c49a92b2a7531bb9f03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29e38b721c7404f8b859746b4133c2f" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 35371 } } peers { permanent_uuid: "8a2c6fb9f21f4c49a92b2a7531bb9f03" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35807 } } peers { permanent_uuid: "34758cb49b4f4778b4bfc6d97cd36000" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 44231 } }
I20260926 15:07:23.036587 27386 raft_consensus.cc:374] T a2239dc1e1ae4de78796b7270b4900c2 P 8a2c6fb9f21f4c49a92b2a7531bb9f03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:23.036680 27386 raft_consensus.cc:729] T a2239dc1e1ae4de78796b7270b4900c2 P 8a2c6fb9f21f4c49a92b2a7531bb9f03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a2c6fb9f21f4c49a92b2a7531bb9f03, State: Initialized, Role: FOLLOWER
I20260926 15:07:23.036939 27385 ts_tablet_manager.cc:1428] T a2239dc1e1ae4de78796b7270b4900c2 P 34758cb49b4f4778b4bfc6d97cd36000: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260926 15:07:23.037329 27386 consensus_queue.cc:260] T a2239dc1e1ae4de78796b7270b4900c2 P 8a2c6fb9f21f4c49a92b2a7531bb9f03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29e38b721c7404f8b859746b4133c2f" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 35371 } } peers { permanent_uuid: "8a2c6fb9f21f4c49a92b2a7531bb9f03" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35807 } } peers { permanent_uuid: "34758cb49b4f4778b4bfc6d97cd36000" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 44231 } }
I20260926 15:07:23.038411 27365 heartbeater.cc:503] Master 127.9.123.126:39593 was elected leader, sending a full tablet report...
I20260926 15:07:23.039170 27386 ts_tablet_manager.cc:1428] T a2239dc1e1ae4de78796b7270b4900c2 P 8a2c6fb9f21f4c49a92b2a7531bb9f03: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260926 15:07:23.070240 27389 raft_consensus.cc:482] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:07:23.070572 27389 raft_consensus.cc:504] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29e38b721c7404f8b859746b4133c2f" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 35371 } } peers { permanent_uuid: "8a2c6fb9f21f4c49a92b2a7531bb9f03" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35807 } } peers { permanent_uuid: "34758cb49b4f4778b4bfc6d97cd36000" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 44231 } }
I20260926 15:07:23.072952 27389 leader_election.cc:288] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a2c6fb9f21f4c49a92b2a7531bb9f03 (127.9.123.67:35807), 34758cb49b4f4778b4bfc6d97cd36000 (127.9.123.66:44231)
W20260926 15:07:23.079464 27098 tablet.cc:2369] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:07:23.082055 27382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:23.073611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:44559 (local address 127.9.123.66:44231)
0926 15:07:23.073827 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:23.073833 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:23.075134 (+ 1301us) server_negotiation.cc:407] Connection header received
0926 15:07:23.075704 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:23.075711 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:23.075791 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:23.075878 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:23.076444 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:23.077457 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:23.079903 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:23.081169 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:23.081340 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:23.081417 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:23.081719 (+ 302us) server_negotiation.cc:299] Negotiation successful
0926 15:07:23.081858 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 15:07:23.082302 27393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:23.074362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:44231 (local address 127.9.123.65:44559)
0926 15:07:23.075033 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:23.075051 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:23.075498 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:23.075911 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:23.075919 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:23.075944 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:23.076281 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:23.076291 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:23.077588 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:23.077593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:23.079645 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:23.079657 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:23.081404 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:23.081409 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:23.081543 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:23.081573 (+ 30us) client_negotiation.cc:769] Sending connection context
0926 15:07:23.081664 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 15:07:23.081817 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"spinlock_wait_cycles":404864,"thread_start_us":700,"threads_started":1}
I20260926 15:07:23.082650 27379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:23.073583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:60637 (local address 127.9.123.67:35807)
0926 15:07:23.073787 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:23.073794 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:23.074604 (+ 810us) server_negotiation.cc:407] Connection header received
0926 15:07:23.074813 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:23.074818 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:23.074891 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:23.074967 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:23.076090 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:23.077179 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:23.080391 (+ 3212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:23.081603 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:23.081761 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:23.081827 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:23.082185 (+ 358us) server_negotiation.cc:299] Negotiation successful
0926 15:07:23.082312 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 15:07:23.082719 27392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:23.073373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:35807 (local address 127.9.123.65:60637)
0926 15:07:23.074483 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:23.074503 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:23.074661 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:23.075008 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:23.075012 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:23.075396 (+ 384us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:23.075952 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:23.075968 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:23.077307 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:23.077311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:23.079638 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:23.079657 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:23.081828 (+ 2171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:23.081833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:23.082007 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:23.082040 (+ 33us) client_negotiation.cc:769] Sending connection context
0926 15:07:23.082137 (+ 97us) client_negotiation.cc:240] Negotiation successful
0926 15:07:23.082285 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":976,"mutex_wait_us":1,"spinlock_wait_cycles":1792,"thread_start_us":756,"threads_started":1}
I20260926 15:07:23.083415 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2239dc1e1ae4de78796b7270b4900c2" candidate_uuid: "f29e38b721c7404f8b859746b4133c2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34758cb49b4f4778b4bfc6d97cd36000" is_pre_election: true
I20260926 15:07:23.083635 27320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2239dc1e1ae4de78796b7270b4900c2" candidate_uuid: "f29e38b721c7404f8b859746b4133c2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a2c6fb9f21f4c49a92b2a7531bb9f03" is_pre_election: true
I20260926 15:07:23.083900 27186 raft_consensus.cc:2454] T a2239dc1e1ae4de78796b7270b4900c2 P 34758cb49b4f4778b4bfc6d97cd36000 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f29e38b721c7404f8b859746b4133c2f in term 0.
I20260926 15:07:23.084108 27320 raft_consensus.cc:2454] T a2239dc1e1ae4de78796b7270b4900c2 P 8a2c6fb9f21f4c49a92b2a7531bb9f03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f29e38b721c7404f8b859746b4133c2f in term 0.
I20260926 15:07:23.084589 26988 leader_election.cc:302] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34758cb49b4f4778b4bfc6d97cd36000, f29e38b721c7404f8b859746b4133c2f; no voters:
I20260926 15:07:23.085220 27389 raft_consensus.cc:2781] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:07:23.085397 27389 raft_consensus.cc:482] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:07:23.085498 27389 raft_consensus.cc:3037] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:23.087918 27389 raft_consensus.cc:504] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29e38b721c7404f8b859746b4133c2f" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 35371 } } peers { permanent_uuid: "8a2c6fb9f21f4c49a92b2a7531bb9f03" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35807 } } peers { permanent_uuid: "34758cb49b4f4778b4bfc6d97cd36000" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 44231 } }
I20260926 15:07:23.088587 27389 leader_election.cc:288] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f [CANDIDATE]: Term 1 election: Requested vote from peers 8a2c6fb9f21f4c49a92b2a7531bb9f03 (127.9.123.67:35807), 34758cb49b4f4778b4bfc6d97cd36000 (127.9.123.66:44231)
I20260926 15:07:23.089169 27320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2239dc1e1ae4de78796b7270b4900c2" candidate_uuid: "f29e38b721c7404f8b859746b4133c2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a2c6fb9f21f4c49a92b2a7531bb9f03"
I20260926 15:07:23.089192 27186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2239dc1e1ae4de78796b7270b4900c2" candidate_uuid: "f29e38b721c7404f8b859746b4133c2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34758cb49b4f4778b4bfc6d97cd36000"
I20260926 15:07:23.089383 27320 raft_consensus.cc:3037] T a2239dc1e1ae4de78796b7270b4900c2 P 8a2c6fb9f21f4c49a92b2a7531bb9f03 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:23.089385 27186 raft_consensus.cc:3037] T a2239dc1e1ae4de78796b7270b4900c2 P 34758cb49b4f4778b4bfc6d97cd36000 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:23.091629 27320 raft_consensus.cc:2454] T a2239dc1e1ae4de78796b7270b4900c2 P 8a2c6fb9f21f4c49a92b2a7531bb9f03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f29e38b721c7404f8b859746b4133c2f in term 1.
I20260926 15:07:23.091651 27186 raft_consensus.cc:2454] T a2239dc1e1ae4de78796b7270b4900c2 P 34758cb49b4f4778b4bfc6d97cd36000 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f29e38b721c7404f8b859746b4133c2f in term 1.
I20260926 15:07:23.092103 26985 leader_election.cc:302] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a2c6fb9f21f4c49a92b2a7531bb9f03, f29e38b721c7404f8b859746b4133c2f; no voters:
I20260926 15:07:23.092440 27389 raft_consensus.cc:2781] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:07:23.092881 27389 raft_consensus.cc:686] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f [term 1 LEADER]: Becoming Leader. State: Replica: f29e38b721c7404f8b859746b4133c2f, State: Running, Role: LEADER
I20260926 15:07:23.093627 27389 consensus_queue.cc:237] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29e38b721c7404f8b859746b4133c2f" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 35371 } } peers { permanent_uuid: "8a2c6fb9f21f4c49a92b2a7531bb9f03" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35807 } } peers { permanent_uuid: "34758cb49b4f4778b4bfc6d97cd36000" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 44231 } }
I20260926 15:07:23.098134 26907 catalog_manager.cc:5495] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f reported cstate change: term changed from 0 to 1, leader changed from <none> to f29e38b721c7404f8b859746b4133c2f (127.9.123.65). New cstate: current_term: 1 leader_uuid: "f29e38b721c7404f8b859746b4133c2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f29e38b721c7404f8b859746b4133c2f" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 35371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a2c6fb9f21f4c49a92b2a7531bb9f03" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34758cb49b4f4778b4bfc6d97cd36000" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 44231 } health_report { overall_health: UNKNOWN } } }
W20260926 15:07:23.124583 27232 tablet.cc:2369] T a2239dc1e1ae4de78796b7270b4900c2 P 34758cb49b4f4778b4bfc6d97cd36000: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 15:07:23.140925 27366 tablet.cc:2369] T a2239dc1e1ae4de78796b7270b4900c2 P 8a2c6fb9f21f4c49a92b2a7531bb9f03: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:07:23.142616 27380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:23.136950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52012 (local address 127.9.123.65:35371)
0926 15:07:23.137223 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:23.137227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:23.137246 (+ 19us) server_negotiation.cc:407] Connection header received
0926 15:07:23.137335 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:23.137339 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:23.137419 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:23.137571 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:07:23.137997 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:23.138862 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:23.140371 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:23.140781 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:23.140910 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:23.141499 (+ 589us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:07:23.142042 (+ 543us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:07:23.142195 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:23.142276 (+ 81us) server_negotiation.cc:299] Negotiation successful
0926 15:07:23.142363 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260926 15:07:23.150769 27186 raft_consensus.cc:1261] T a2239dc1e1ae4de78796b7270b4900c2 P 34758cb49b4f4778b4bfc6d97cd36000 [term 1 FOLLOWER]: Refusing update from remote peer f29e38b721c7404f8b859746b4133c2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:07:23.150765 27320 raft_consensus.cc:1261] T a2239dc1e1ae4de78796b7270b4900c2 P 8a2c6fb9f21f4c49a92b2a7531bb9f03 [term 1 FOLLOWER]: Refusing update from remote peer f29e38b721c7404f8b859746b4133c2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:07:23.151844 27394 consensus_queue.cc:1035] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "34758cb49b4f4778b4bfc6d97cd36000" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 44231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:07:23.152252 27389 consensus_queue.cc:1035] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a2c6fb9f21f4c49a92b2a7531bb9f03" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:07:23.167169 27399 mvcc.cc:205] Tried to move back new op lower bound from 7333622755935899648 to 7333622755715440640. Current Snapshot: MvccSnapshot[applied={T|T < 7333622755935899648}]
I20260926 15:07:23.170699 27398 mvcc.cc:205] Tried to move back new op lower bound from 7333622755935899648 to 7333622755715440640. Current Snapshot: MvccSnapshot[applied={T|T < 7333622755935899648}]
I20260926 15:07:23.181964 26906 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:07:23.182363 26906 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:07:23.186882 26906 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 859b69e557e34c31850a5752792620bb: Sending DeleteTablet for 3 replicas of tablet a2239dc1e1ae4de78796b7270b4900c2
I20260926 15:07:23.188372 27300 tablet_service.cc:1514] Processing DeleteTablet for tablet a2239dc1e1ae4de78796b7270b4900c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:07:23 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:42464
I20260926 15:07:23.188750 27166 tablet_service.cc:1514] Processing DeleteTablet for tablet a2239dc1e1ae4de78796b7270b4900c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:07:23 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:34788
I20260926 15:07:23.189241 27032 tablet_service.cc:1514] Processing DeleteTablet for tablet a2239dc1e1ae4de78796b7270b4900c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:07:23 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:52010
I20260926 15:07:23.190200 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 26969
I20260926 15:07:23.201262 27408 tablet_replica.cc:330] T a2239dc1e1ae4de78796b7270b4900c2 P 34758cb49b4f4778b4bfc6d97cd36000: stopping tablet replica
I20260926 15:07:23.201813 27408 raft_consensus.cc:2229] T a2239dc1e1ae4de78796b7270b4900c2 P 34758cb49b4f4778b4bfc6d97cd36000 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:07:23.202415 27408 raft_consensus.cc:2258] T a2239dc1e1ae4de78796b7270b4900c2 P 34758cb49b4f4778b4bfc6d97cd36000 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:07:23.203430 27409 tablet_replica.cc:330] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f: stopping tablet replica
I20260926 15:07:23.203883 27409 raft_consensus.cc:2229] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:07:23.204610 27409 raft_consensus.cc:2258] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:07:23.205472 27407 tablet_replica.cc:330] T a2239dc1e1ae4de78796b7270b4900c2 P 8a2c6fb9f21f4c49a92b2a7531bb9f03: stopping tablet replica
I20260926 15:07:23.205967 27407 raft_consensus.cc:2229] T a2239dc1e1ae4de78796b7270b4900c2 P 8a2c6fb9f21f4c49a92b2a7531bb9f03 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:07:23.206477 27407 raft_consensus.cc:2258] T a2239dc1e1ae4de78796b7270b4900c2 P 8a2c6fb9f21f4c49a92b2a7531bb9f03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:07:23.206972 27408 ts_tablet_manager.cc:1905] T a2239dc1e1ae4de78796b7270b4900c2 P 34758cb49b4f4778b4bfc6d97cd36000: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:07:23.208807 27407 ts_tablet_manager.cc:1905] T a2239dc1e1ae4de78796b7270b4900c2 P 8a2c6fb9f21f4c49a92b2a7531bb9f03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:07:23.212457 27409 ts_tablet_manager.cc:1905] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:07:23.214522 27408 ts_tablet_manager.cc:1918] T a2239dc1e1ae4de78796b7270b4900c2 P 34758cb49b4f4778b4bfc6d97cd36000: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:07:23.214704 27408 log.cc:1196] T a2239dc1e1ae4de78796b7270b4900c2 P 34758cb49b4f4778b4bfc6d97cd36000: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/a2239dc1e1ae4de78796b7270b4900c2
I20260926 15:07:23.215200 27408 ts_tablet_manager.cc:1939] T a2239dc1e1ae4de78796b7270b4900c2 P 34758cb49b4f4778b4bfc6d97cd36000: Deleting consensus metadata
I20260926 15:07:23.215646 27407 ts_tablet_manager.cc:1918] T a2239dc1e1ae4de78796b7270b4900c2 P 8a2c6fb9f21f4c49a92b2a7531bb9f03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:07:23.215821 27407 log.cc:1196] T a2239dc1e1ae4de78796b7270b4900c2 P 8a2c6fb9f21f4c49a92b2a7531bb9f03: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/a2239dc1e1ae4de78796b7270b4900c2
I20260926 15:07:23.216315 27407 ts_tablet_manager.cc:1939] T a2239dc1e1ae4de78796b7270b4900c2 P 8a2c6fb9f21f4c49a92b2a7531bb9f03: Deleting consensus metadata
I20260926 15:07:23.217067 26894 catalog_manager.cc:4841] TS 34758cb49b4f4778b4bfc6d97cd36000 (127.9.123.66:44231): tablet a2239dc1e1ae4de78796b7270b4900c2 (table test-table [id=afe7eae7661d4a4ea65598e138ab3c41]) successfully deleted
I20260926 15:07:23.218214 26891 catalog_manager.cc:4841] TS 8a2c6fb9f21f4c49a92b2a7531bb9f03 (127.9.123.67:35807): tablet a2239dc1e1ae4de78796b7270b4900c2 (table test-table [id=afe7eae7661d4a4ea65598e138ab3c41]) successfully deleted
Sep 26 15:07:23 dist-test-slave-7153 krb5kdc[26862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.65@KRBTEST.COM
I20260926 15:07:23.222806 27409 ts_tablet_manager.cc:1918] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:07:23.223131 27409 log.cc:1196] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790434939095847-9709-0/minicluster-data/ts-0/wal/wals/a2239dc1e1ae4de78796b7270b4900c2
I20260926 15:07:23.223776 27409 ts_tablet_manager.cc:1939] T a2239dc1e1ae4de78796b7270b4900c2 P f29e38b721c7404f8b859746b4133c2f: Deleting consensus metadata
I20260926 15:07:23.225898 26894 catalog_manager.cc:4841] TS f29e38b721c7404f8b859746b4133c2f (127.9.123.65:35371): tablet a2239dc1e1ae4de78796b7270b4900c2 (table test-table [id=afe7eae7661d4a4ea65598e138ab3c41]) successfully deleted
I20260926 15:07:23.228938 27380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:23.191665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52028 (local address 127.9.123.65:35371)
0926 15:07:23.191872 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:23.191878 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:23.202979 (+ 11101us) server_negotiation.cc:407] Connection header received
0926 15:07:23.203298 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:23.203305 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:23.203384 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:23.203472 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:23.205527 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:23.206603 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:23.208641 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:23.209330 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:23.209439 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:23.221462 (+ 12023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:23.221484 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:23.221506 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:23.221564 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:23.225718 (+ 4154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:23.226684 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:23.226692 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:23.226694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:23.226852 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:23.227278 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:23.227282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:23.227284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:23.227658 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:23.227956 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:23.228320 (+ 364us) server_negotiation.cc:299] Negotiation successful
0926 15:07:23.228584 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 15:07:23.229698 27092 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:23.330387 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 26969
I20260926 15:07:23.346053 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 27103
Sep 26 15:07:23 dist-test-slave-7153 krb5kdc[26862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.66@KRBTEST.COM
I20260926 15:07:23.361129 27382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:23.346917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34790 (local address 127.9.123.66:44231)
0926 15:07:23.347098 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:23.347102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:23.347120 (+ 18us) server_negotiation.cc:407] Connection header received
0926 15:07:23.347224 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:23.347228 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:23.347289 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:23.347379 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:23.348674 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:23.349540 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:23.350952 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:23.351355 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:23.351463 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:23.355789 (+ 4326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:23.355804 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:23.355819 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:23.355854 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:23.358456 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:23.359110 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:23.359117 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:23.359119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:23.359240 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:23.359598 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:23.359601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:23.359603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:23.359990 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:23.360283 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:23.360554 (+ 271us) server_negotiation.cc:299] Negotiation successful
0926 15:07:23.360774 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 15:07:23.361866 27226 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:23.449855 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 27103
I20260926 15:07:23.464900 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 27237
Sep 26 15:07:23 dist-test-slave-7153 krb5kdc[26862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.67@KRBTEST.COM
I20260926 15:07:23.482338 27379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:23.465829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42468 (local address 127.9.123.67:35807)
0926 15:07:23.466066 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:23.466078 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:23.466100 (+ 22us) server_negotiation.cc:407] Connection header received
0926 15:07:23.466159 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:23.466163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:23.466229 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:23.466318 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:23.467684 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:23.468541 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:23.470068 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:23.470660 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:23.470896 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:23.475981 (+ 5085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:23.476004 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:23.476023 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:23.476070 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:23.479054 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:23.479998 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:23.480010 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:23.480013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:23.480179 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:23.480653 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:23.480658 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:23.480660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:23.481091 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:23.481405 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:23.481764 (+ 359us) server_negotiation.cc:299] Negotiation successful
0926 15:07:23.481968 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260926 15:07:23.483129 27360 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:23.582240 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 27237
I20260926 15:07:23.595929 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 26875
I20260926 15:07:23.596973 26937 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:23.708158 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 26875
2026-09-26T15:07:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4198 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-26T15:07:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:07:23Z Disabled control of system clock
I20260926 15:07:23.778805 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:40271
--webserver_interface=127.9.123.126
--webserver_port=0
--builtin_ntp_servers=127.9.123.84:37879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:40271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 15:07:24.112620 27418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:24.112967 27418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:24.113091 27418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:24.120633 27418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:07:24.120767 27418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:24.120838 27418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:24.120918 27418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:07:24.120978 27418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:07:24.130891 27418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:37879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:40271
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:40271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:24.132220 27418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:24.134079 27418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:24.145712 27423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:24.145669 27424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:24.145766 27426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:24.146391 27418 server_base.cc:1034] running on GCE node
I20260926 15:07:24.147156 27418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:24.148588 27418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:24.149811 27418 hybrid_clock.cc:648] HybridClock initialized: now 1790435244149752 us; error 48 us; skew 500 ppm
I20260926 15:07:24.150198 27418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:07:24.152549 27418 webserver.cc:460] Webserver started at http://127.9.123.126:37203/ using document root <none> and password file <none>
I20260926 15:07:24.153290 27418 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:24.153386 27418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:24.153656 27418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:24.156646 27418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "5d658ff58a23463ab9c62bac5ab9615d"
format_stamp: "Formatted at 2026-09-26 15:07:24 on dist-test-slave-7153"
server_key: "4d1d3e18d6b7572db22521213d6083c2"
server_key_iv: "0d170a648e3ef8d282cbc2da514c6545"
server_key_version: "encryptionkey@0"
I20260926 15:07:24.157526 27418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "5d658ff58a23463ab9c62bac5ab9615d"
format_stamp: "Formatted at 2026-09-26 15:07:24 on dist-test-slave-7153"
server_key: "4d1d3e18d6b7572db22521213d6083c2"
server_key_iv: "0d170a648e3ef8d282cbc2da514c6545"
server_key_version: "encryptionkey@0"
I20260926 15:07:24.162895 27418 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260926 15:07:24.166679 27432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:24.168135 27418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:07:24.168299 27418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "5d658ff58a23463ab9c62bac5ab9615d"
format_stamp: "Formatted at 2026-09-26 15:07:24 on dist-test-slave-7153"
server_key: "4d1d3e18d6b7572db22521213d6083c2"
server_key_iv: "0d170a648e3ef8d282cbc2da514c6545"
server_key_version: "encryptionkey@0"
I20260926 15:07:24.168499 27418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:24.190060 27418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:24.190936 27418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:24.191191 27418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:24.208403 27418 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:40271
I20260926 15:07:24.208427 27483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:40271 every 8 connection(s)
I20260926 15:07:24.209832 27418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:07:24.211537 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 27418
I20260926 15:07:24.211743 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:07:24.212294 9709 external_mini_cluster.cc:1428] Setting key 67371432fc9d7d07980f0b0b174aa9e8
I20260926 15:07:24.214576 27484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:24.227010 27487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:24.214291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55322 (local address 127.9.123.126:40271)
0926 15:07:24.215302 (+ 1011us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:24.215313 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:24.215355 (+ 42us) server_negotiation.cc:407] Connection header received
0926 15:07:24.216997 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:24.217021 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:24.217764 (+ 743us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:24.218215 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:24.218835 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:24.220279 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:24.222313 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:24.223029 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:24.223182 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:24.224489 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:24.224501 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:24.224517 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:24.225188 (+ 671us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:24.225287 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:24.225312 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:24.225411 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:24.225641 (+ 230us) server_negotiation.cc:299] Negotiation successful
0926 15:07:24.226015 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":380,"threads_started":1}
I20260926 15:07:24.227840 27484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d658ff58a23463ab9c62bac5ab9615d: Bootstrap starting.
I20260926 15:07:24.231096 27484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d658ff58a23463ab9c62bac5ab9615d: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:24.232311 27484 log.cc:824] T 00000000000000000000000000000000 P 5d658ff58a23463ab9c62bac5ab9615d: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:24.235486 27484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d658ff58a23463ab9c62bac5ab9615d: No bootstrap required, opened a new log
I20260926 15:07:24.243295 27484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d658ff58a23463ab9c62bac5ab9615d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d658ff58a23463ab9c62bac5ab9615d" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 40271 } }
I20260926 15:07:24.243719 27484 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d658ff58a23463ab9c62bac5ab9615d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:24.243816 27484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d658ff58a23463ab9c62bac5ab9615d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d658ff58a23463ab9c62bac5ab9615d, State: Initialized, Role: FOLLOWER
I20260926 15:07:24.244436 27484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d658ff58a23463ab9c62bac5ab9615d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d658ff58a23463ab9c62bac5ab9615d" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 40271 } }
I20260926 15:07:24.244674 27484 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d658ff58a23463ab9c62bac5ab9615d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:07:24.244797 27484 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d658ff58a23463ab9c62bac5ab9615d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:07:24.244969 27484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d658ff58a23463ab9c62bac5ab9615d [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:24.247553 27484 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d658ff58a23463ab9c62bac5ab9615d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d658ff58a23463ab9c62bac5ab9615d" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 40271 } }
I20260926 15:07:24.248098 27484 leader_election.cc:302] T 00000000000000000000000000000000 P 5d658ff58a23463ab9c62bac5ab9615d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d658ff58a23463ab9c62bac5ab9615d; no voters:
I20260926 15:07:24.248695 27484 leader_election.cc:288] T 00000000000000000000000000000000 P 5d658ff58a23463ab9c62bac5ab9615d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:07:24.248858 27489 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d658ff58a23463ab9c62bac5ab9615d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:07:24.249594 27489 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d658ff58a23463ab9c62bac5ab9615d [term 1 LEADER]: Becoming Leader. State: Replica: 5d658ff58a23463ab9c62bac5ab9615d, State: Running, Role: LEADER
I20260926 15:07:24.250124 27489 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d658ff58a23463ab9c62bac5ab9615d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d658ff58a23463ab9c62bac5ab9615d" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 40271 } }
I20260926 15:07:24.250788 27484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d658ff58a23463ab9c62bac5ab9615d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:07:24.253150 27491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d658ff58a23463ab9c62bac5ab9615d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d658ff58a23463ab9c62bac5ab9615d. Latest consensus state: current_term: 1 leader_uuid: "5d658ff58a23463ab9c62bac5ab9615d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d658ff58a23463ab9c62bac5ab9615d" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 40271 } } }
I20260926 15:07:24.253121 27490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d658ff58a23463ab9c62bac5ab9615d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d658ff58a23463ab9c62bac5ab9615d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d658ff58a23463ab9c62bac5ab9615d" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 40271 } } }
I20260926 15:07:24.253633 27491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d658ff58a23463ab9c62bac5ab9615d [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:24.253733 27490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d658ff58a23463ab9c62bac5ab9615d [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:24.255290 27496 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:07:24.260877 27496 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:07:24.271423 27496 catalog_manager.cc:1347] Generated new cluster ID: c045437289a54d02aeb3d3a0dc67ac23
I20260926 15:07:24.271561 27496 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:07:24.310345 27496 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:07:24.311666 27496 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:07:24.324723 27496 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d658ff58a23463ab9c62bac5ab9615d: Generated new TSK 0
I20260926 15:07:24.325677 27496 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:07:24.339509 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:40271
--builtin_ntp_servers=127.9.123.84:37879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 15:07:24.689527 27508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:24.689824 27508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:24.689936 27508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:24.697196 27508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:24.697310 27508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:24.697418 27508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:07:24.706651 27508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:37879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:40271
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:24.707939 27508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:24.709834 27508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:24.723397 27513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:24.723456 27516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:24.723516 27514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:24.724560 27508 server_base.cc:1034] running on GCE node
I20260926 15:07:24.725284 27508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:24.726385 27508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:24.727660 27508 hybrid_clock.cc:648] HybridClock initialized: now 1790435244727593 us; error 58 us; skew 500 ppm
I20260926 15:07:24.728092 27508 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:07:24.730628 27508 webserver.cc:460] Webserver started at http://127.9.123.65:36749/ using document root <none> and password file <none>
I20260926 15:07:24.731348 27508 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:24.731441 27508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:24.731722 27508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:24.734843 27508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "27b9e914b7bc4d56b6a9ef41669e4d8e"
format_stamp: "Formatted at 2026-09-26 15:07:24 on dist-test-slave-7153"
server_key: "d572cc990262b7ddc4e8da13eb44beb9"
server_key_iv: "a7daf977cc62c6a9e1a5336a74899496"
server_key_version: "encryptionkey@0"
I20260926 15:07:24.735625 27508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "27b9e914b7bc4d56b6a9ef41669e4d8e"
format_stamp: "Formatted at 2026-09-26 15:07:24 on dist-test-slave-7153"
server_key: "d572cc990262b7ddc4e8da13eb44beb9"
server_key_iv: "a7daf977cc62c6a9e1a5336a74899496"
server_key_version: "encryptionkey@0"
I20260926 15:07:24.741034 27508 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260926 15:07:24.744899 27522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:24.746385 27508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:07:24.746546 27508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "27b9e914b7bc4d56b6a9ef41669e4d8e"
format_stamp: "Formatted at 2026-09-26 15:07:24 on dist-test-slave-7153"
server_key: "d572cc990262b7ddc4e8da13eb44beb9"
server_key_iv: "a7daf977cc62c6a9e1a5336a74899496"
server_key_version: "encryptionkey@0"
I20260926 15:07:24.746762 27508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:24.777770 27508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:24.778651 27508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:24.778939 27508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:24.779973 27508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:07:24.781520 27508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:07:24.781610 27508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:24.781719 27508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:07:24.781790 27508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:24.812695 27508 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:38973
I20260926 15:07:24.812770 27634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:38973 every 8 connection(s)
I20260926 15:07:24.813982 27508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:07:24.814381 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 27508
I20260926 15:07:24.814543 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:07:24.815145 9709 external_mini_cluster.cc:1428] Setting key ff58e6b328489df7eec2f039c16e9493
I20260926 15:07:24.818506 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:40271
--builtin_ntp_servers=127.9.123.84:37879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 15:07:24.827976 27637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:24.816812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:33703 (local address 127.9.123.126:40271)
0926 15:07:24.817263 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:24.817268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:24.818842 (+ 1574us) server_negotiation.cc:407] Connection header received
0926 15:07:24.820696 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:24.820702 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:24.820790 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:24.820884 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:24.822089 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:24.822851 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:24.824630 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:24.825197 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:24.825319 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:24.827091 (+ 1772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:24.827097 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:24.827103 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:24.827127 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:24.827177 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:24.827191 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:24.827282 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:24.827521 (+ 239us) server_negotiation.cc:299] Negotiation successful
0926 15:07:24.827622 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":213,"threads_started":1}
I20260926 15:07:24.828917 27638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:24.817563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:40271 (local address 127.9.123.65:33703)
0926 15:07:24.818601 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:24.818649 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:24.820416 (+ 1767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:24.821189 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:24.821198 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:24.821297 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:07:24.821917 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:24.821935 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:24.823000 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:24.823004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:24.824489 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:24.824502 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:24.825606 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:24.825613 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:24.825903 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:24.826680 (+ 777us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 15:07:24.826684 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:07:24.826741 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 15:07:24.826875 (+ 134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:07:24.827301 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:07:24.827310 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:07:24.827316 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 15:07:24.827494 (+ 178us) client_negotiation.cc:240] Negotiation successful
0926 15:07:24.827819 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":340,"threads_started":1}
I20260926 15:07:24.830478 27635 heartbeater.cc:347] Connected to a master server at 127.9.123.126:40271
I20260926 15:07:24.830789 27635 heartbeater.cc:464] Registering TS with master...
I20260926 15:07:24.831586 27635 heartbeater.cc:511] Master 127.9.123.126:40271 requested a full tablet report, sending...
I20260926 15:07:24.833856 27449 ts_manager.cc:194] Registered new tserver with Master: 27b9e914b7bc4d56b6a9ef41669e4d8e (127.9.123.65:38973)
I20260926 15:07:24.835322 27449 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.123.65:33703
W20260926 15:07:25.148190 27639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:25.148469 27639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:25.148561 27639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:25.156083 27639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:25.156193 27639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:25.156307 27639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:07:25.165544 27639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:37879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:40271
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:25.166760 27639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:25.168469 27639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:25.181885 27644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:25.181953 27645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:25.181978 27647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:25.182487 27639 server_base.cc:1034] running on GCE node
I20260926 15:07:25.183171 27639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:25.184094 27639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:25.185286 27639 hybrid_clock.cc:648] HybridClock initialized: now 1790435245185237 us; error 42 us; skew 500 ppm
I20260926 15:07:25.185618 27639 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:07:25.187850 27639 webserver.cc:460] Webserver started at http://127.9.123.66:43083/ using document root <none> and password file <none>
I20260926 15:07:25.188553 27639 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:25.188668 27639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:25.188974 27639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:25.191972 27639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "283297be26ff46d2a8a7601fc72676ac"
format_stamp: "Formatted at 2026-09-26 15:07:25 on dist-test-slave-7153"
server_key: "23f4e4a5114c2409571cb51bf7fce883"
server_key_iv: "73a7349949eccdd66c314719bddaefbf"
server_key_version: "encryptionkey@0"
I20260926 15:07:25.192768 27639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "283297be26ff46d2a8a7601fc72676ac"
format_stamp: "Formatted at 2026-09-26 15:07:25 on dist-test-slave-7153"
server_key: "23f4e4a5114c2409571cb51bf7fce883"
server_key_iv: "73a7349949eccdd66c314719bddaefbf"
server_key_version: "encryptionkey@0"
I20260926 15:07:25.198038 27639 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260926 15:07:25.201645 27653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:25.203138 27639 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260926 15:07:25.203295 27639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "283297be26ff46d2a8a7601fc72676ac"
format_stamp: "Formatted at 2026-09-26 15:07:25 on dist-test-slave-7153"
server_key: "23f4e4a5114c2409571cb51bf7fce883"
server_key_iv: "73a7349949eccdd66c314719bddaefbf"
server_key_version: "encryptionkey@0"
I20260926 15:07:25.203517 27639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:25.224009 27639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:25.224910 27639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:25.225270 27639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:25.226274 27639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:07:25.227766 27639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:07:25.227875 27639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:25.227978 27639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:07:25.228044 27639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:25.258486 27639 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:44789
I20260926 15:07:25.258559 27765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:44789 every 8 connection(s)
I20260926 15:07:25.259801 27639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:07:25.263700 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 27639
I20260926 15:07:25.263881 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:07:25.264370 9709 external_mini_cluster.cc:1428] Setting key 09dece8f3b660e237d369f31ddd6c2a9
I20260926 15:07:25.267099 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:40271
--builtin_ntp_servers=127.9.123.84:37879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 15:07:25.272934 27637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:25.262452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:35595 (local address 127.9.123.126:40271)
0926 15:07:25.262674 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:25.262680 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:25.264192 (+ 1512us) server_negotiation.cc:407] Connection header received
0926 15:07:25.266121 (+ 1929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:25.266125 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:25.266185 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:25.266270 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:25.267404 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.268426 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.270138 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.270666 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.270783 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:25.272148 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:25.272154 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:25.272159 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:25.272184 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:25.272220 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:25.272234 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:25.272314 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:25.272513 (+ 199us) server_negotiation.cc:299] Negotiation successful
0926 15:07:25.272622 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 15:07:25.274051 27768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:25.263013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:40271 (local address 127.9.123.66:35595)
0926 15:07:25.264032 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:25.264071 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:25.265855 (+ 1784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:25.266474 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:25.266483 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:25.266584 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:07:25.267175 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:25.267200 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.268574 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.268578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:25.269974 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:25.270009 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.270889 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.270894 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:25.271143 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:25.271814 (+ 671us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 15:07:25.271817 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:07:25.271861 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 15:07:25.271950 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:07:25.272323 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:07:25.272333 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:07:25.272339 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 15:07:25.272502 (+ 163us) client_negotiation.cc:240] Negotiation successful
0926 15:07:25.272899 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":307,"threads_started":1}
I20260926 15:07:25.275518 27766 heartbeater.cc:347] Connected to a master server at 127.9.123.126:40271
I20260926 15:07:25.275811 27766 heartbeater.cc:464] Registering TS with master...
I20260926 15:07:25.276557 27766 heartbeater.cc:511] Master 127.9.123.126:40271 requested a full tablet report, sending...
I20260926 15:07:25.278016 27449 ts_manager.cc:194] Registered new tserver with Master: 283297be26ff46d2a8a7601fc72676ac (127.9.123.66:44789)
I20260926 15:07:25.278831 27449 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.123.66:35595
W20260926 15:07:25.599931 27769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:25.600243 27769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:25.600350 27769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:25.608004 27769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:25.608124 27769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:25.608260 27769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:07:25.618192 27769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:37879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:40271
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:25.619547 27769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:25.621578 27769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:25.635150 27774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:25.635092 27777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:25.635056 27775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:25.635768 27769 server_base.cc:1034] running on GCE node
I20260926 15:07:25.636440 27769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:25.637447 27769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:25.638686 27769 hybrid_clock.cc:648] HybridClock initialized: now 1790435245638628 us; error 46 us; skew 500 ppm
I20260926 15:07:25.639020 27769 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:07:25.641408 27769 webserver.cc:460] Webserver started at http://127.9.123.67:36107/ using document root <none> and password file <none>
I20260926 15:07:25.642159 27769 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:25.642261 27769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:25.642567 27769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:25.645485 27769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "93508c72fd2d43379417cbf4d438a74b"
format_stamp: "Formatted at 2026-09-26 15:07:25 on dist-test-slave-7153"
server_key: "94f954342a18b84c317db0cc1489c2ab"
server_key_iv: "6fddf2e636e28d32b34d97320ac381a1"
server_key_version: "encryptionkey@0"
I20260926 15:07:25.646260 27769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "93508c72fd2d43379417cbf4d438a74b"
format_stamp: "Formatted at 2026-09-26 15:07:25 on dist-test-slave-7153"
server_key: "94f954342a18b84c317db0cc1489c2ab"
server_key_iv: "6fddf2e636e28d32b34d97320ac381a1"
server_key_version: "encryptionkey@0"
I20260926 15:07:25.651996 27769 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260926 15:07:25.655999 27783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:25.657646 27769 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:07:25.657819 27769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "93508c72fd2d43379417cbf4d438a74b"
format_stamp: "Formatted at 2026-09-26 15:07:25 on dist-test-slave-7153"
server_key: "94f954342a18b84c317db0cc1489c2ab"
server_key_iv: "6fddf2e636e28d32b34d97320ac381a1"
server_key_version: "encryptionkey@0"
I20260926 15:07:25.658039 27769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:25.683795 27769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:25.684640 27769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:25.684922 27769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:25.685978 27769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:07:25.687508 27769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:07:25.687593 27769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:25.687695 27769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:07:25.687768 27769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:25.717586 27769 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:40789
I20260926 15:07:25.717671 27895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:40789 every 8 connection(s)
I20260926 15:07:25.718888 27769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:07:25.722164 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 27769
I20260926 15:07:25.722337 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:07:25.722882 9709 external_mini_cluster.cc:1428] Setting key bed37e1e003292661b579ae63ea3e881
I20260926 15:07:25.731969 27637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:25.721485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:46395 (local address 127.9.123.126:40271)
0926 15:07:25.721650 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:25.721654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:25.723315 (+ 1661us) server_negotiation.cc:407] Connection header received
0926 15:07:25.725582 (+ 2267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:25.725587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:25.725676 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:25.725789 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:25.727086 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.727837 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.729488 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.729900 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.730003 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:25.731277 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:25.731282 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:25.731286 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:25.731306 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:25.731340 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:25.731351 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:25.731424 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:25.731612 (+ 188us) server_negotiation.cc:299] Negotiation successful
0926 15:07:25.731698 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 15:07:25.733047 27898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:25.721980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:40271 (local address 127.9.123.67:46395)
0926 15:07:25.723126 (+ 1146us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:25.723181 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:25.725280 (+ 2099us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:25.726077 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:25.726090 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:25.726244 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:07:25.726905 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:25.726924 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.727994 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.727998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:25.729345 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:25.729358 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.730086 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.730091 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:25.730303 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:25.730944 (+ 641us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 15:07:25.730947 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:07:25.730990 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 15:07:25.731080 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:07:25.731439 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:07:25.731446 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:07:25.731451 (+ 5us) client_negotiation.cc:769] Sending connection context
0926 15:07:25.731597 (+ 146us) client_negotiation.cc:240] Negotiation successful
0926 15:07:25.731960 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":315,"threads_started":1}
I20260926 15:07:25.734423 27896 heartbeater.cc:347] Connected to a master server at 127.9.123.126:40271
I20260926 15:07:25.734701 27896 heartbeater.cc:464] Registering TS with master...
I20260926 15:07:25.735427 27896 heartbeater.cc:511] Master 127.9.123.126:40271 requested a full tablet report, sending...
I20260926 15:07:25.736887 27449 ts_manager.cc:194] Registered new tserver with Master: 93508c72fd2d43379417cbf4d438a74b (127.9.123.67:40789)
I20260926 15:07:25.737907 27449 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.123.67:46395
I20260926 15:07:25.738497 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 15:07:25.749713 27637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:25.742627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55334 (local address 127.9.123.126:40271)
0926 15:07:25.742831 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:25.742836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:25.743299 (+ 463us) server_negotiation.cc:407] Connection header received
0926 15:07:25.743498 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:25.743502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:25.743580 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:25.743673 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:25.744258 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.745316 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.746814 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.747486 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.747618 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:25.748802 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:25.748807 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:25.748811 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:25.748835 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:25.748872 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:25.748884 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:25.748966 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:25.749290 (+ 324us) server_negotiation.cc:299] Negotiation successful
0926 15:07:25.749395 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 15:07:25.753989 27449 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:07:25.760946 27449 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:07:25.796692 27908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:25.774133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:40789 (local address 127.0.0.1:40740)
0926 15:07:25.788155 (+ 14022us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:25.788171 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:25.788327 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:25.789175 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:25.789183 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:25.789219 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:25.789586 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:25.789597 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.791549 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.791554 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:25.793886 (+ 2332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:25.793906 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.795761 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.795767 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:25.795988 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:25.796044 (+ 56us) client_negotiation.cc:769] Sending connection context
0926 15:07:25.796155 (+ 111us) client_negotiation.cc:240] Negotiation successful
0926 15:07:25.796349 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13900,"thread_start_us":10930,"threads_started":1}
I20260926 15:07:25.797557 27911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:25.774382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40740 (local address 127.9.123.67:40789)
0926 15:07:25.787765 (+ 13383us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:25.787772 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:25.788303 (+ 531us) server_negotiation.cc:407] Connection header received
0926 15:07:25.788503 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:25.788513 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:25.788865 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:25.789083 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:25.789761 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.791385 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.794129 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.795569 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.795699 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:25.795996 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:25.796999 (+ 1003us) server_negotiation.cc:299] Negotiation successful
0926 15:07:25.797225 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13232,"thread_start_us":13080,"threads_started":1}
I20260926 15:07:25.798331 27910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:25.774220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34230 (local address 127.9.123.66:44789)
0926 15:07:25.786960 (+ 12740us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:25.786971 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:25.787016 (+ 45us) server_negotiation.cc:407] Connection header received
0926 15:07:25.789881 (+ 2865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:25.789892 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:25.790174 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:25.790357 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:25.791216 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.792766 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.795128 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.796483 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.796619 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:25.797225 (+ 606us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:25.797948 (+ 723us) server_negotiation.cc:299] Negotiation successful
0926 15:07:25.798102 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12540,"thread_start_us":12840,"threads_started":1}
I20260926 15:07:25.798354 27909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:25.773975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:44789 (local address 127.0.0.1:34230)
0926 15:07:25.785513 (+ 11538us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:25.785558 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:25.785800 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:25.790630 (+ 4830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:25.790635 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:25.790662 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:25.790996 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:25.791009 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.792948 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.792953 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:25.794938 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:25.794951 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.797378 (+ 2427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.797386 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:25.797653 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:25.797733 (+ 80us) client_negotiation.cc:769] Sending connection context
0926 15:07:25.797859 (+ 126us) client_negotiation.cc:240] Negotiation successful
0926 15:07:25.797983 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11261,"mutex_wait_us":186,"thread_start_us":14850,"threads_started":1}
I20260926 15:07:25.798596 27912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:25.786073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:38973 (local address 127.0.0.1:38922)
0926 15:07:25.788861 (+ 2788us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:25.788877 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:25.789000 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:25.791024 (+ 2024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:25.791029 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:25.791060 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:25.791368 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:25.791378 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.792848 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.792853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:25.795232 (+ 2379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:25.795247 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.797395 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.797400 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:25.797656 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:25.797737 (+ 81us) client_negotiation.cc:769] Sending connection context
0926 15:07:25.797816 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 15:07:25.797962 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2646,"mutex_wait_us":115,"thread_start_us":2507,"threads_started":1}
I20260926 15:07:25.799983 27913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:25.786346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38922 (local address 127.9.123.65:38973)
0926 15:07:25.790532 (+ 4186us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:25.790540 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:25.790567 (+ 27us) server_negotiation.cc:407] Connection header received
0926 15:07:25.790656 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:25.790662 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:25.790826 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:25.791011 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:25.791522 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.792686 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.795447 (+ 2761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.796901 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.797020 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:25.797280 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:25.798373 (+ 1093us) server_negotiation.cc:299] Negotiation successful
0926 15:07:25.798663 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4044,"thread_start_us":3922,"threads_started":1}
I20260926 15:07:25.803004 27570 tablet_service.cc:1467] Processing CreateTablet for tablet 8accec55724f40e2b3bb1d841ca0de6a (DEFAULT_TABLE table=test-table [id=4dbcc285d2dc451282fe15ef066f1c54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:07:25.802939 27831 tablet_service.cc:1467] Processing CreateTablet for tablet 8accec55724f40e2b3bb1d841ca0de6a (DEFAULT_TABLE table=test-table [id=4dbcc285d2dc451282fe15ef066f1c54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:07:25.803707 27701 tablet_service.cc:1467] Processing CreateTablet for tablet 8accec55724f40e2b3bb1d841ca0de6a (DEFAULT_TABLE table=test-table [id=4dbcc285d2dc451282fe15ef066f1c54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:07:25.804231 27570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8accec55724f40e2b3bb1d841ca0de6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:25.804368 27831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8accec55724f40e2b3bb1d841ca0de6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:25.804955 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8accec55724f40e2b3bb1d841ca0de6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:25.822974 27915 tablet_bootstrap.cc:492] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac: Bootstrap starting.
I20260926 15:07:25.824707 27914 tablet_bootstrap.cc:492] T 8accec55724f40e2b3bb1d841ca0de6a P 27b9e914b7bc4d56b6a9ef41669e4d8e: Bootstrap starting.
I20260926 15:07:25.828076 27915 tablet_bootstrap.cc:654] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:25.829262 27915 log.cc:824] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:25.832062 27915 tablet_bootstrap.cc:492] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac: No bootstrap required, opened a new log
I20260926 15:07:25.832361 27915 ts_tablet_manager.cc:1397] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260926 15:07:25.833009 27916 tablet_bootstrap.cc:492] T 8accec55724f40e2b3bb1d841ca0de6a P 93508c72fd2d43379417cbf4d438a74b: Bootstrap starting.
I20260926 15:07:25.833217 27914 tablet_bootstrap.cc:654] T 8accec55724f40e2b3bb1d841ca0de6a P 27b9e914b7bc4d56b6a9ef41669e4d8e: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:25.834669 27914 log.cc:824] T 8accec55724f40e2b3bb1d841ca0de6a P 27b9e914b7bc4d56b6a9ef41669e4d8e: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:25.837406 27916 tablet_bootstrap.cc:654] T 8accec55724f40e2b3bb1d841ca0de6a P 93508c72fd2d43379417cbf4d438a74b: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:25.837536 27914 tablet_bootstrap.cc:492] T 8accec55724f40e2b3bb1d841ca0de6a P 27b9e914b7bc4d56b6a9ef41669e4d8e: No bootstrap required, opened a new log
I20260926 15:07:25.837785 27914 ts_tablet_manager.cc:1397] T 8accec55724f40e2b3bb1d841ca0de6a P 27b9e914b7bc4d56b6a9ef41669e4d8e: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260926 15:07:25.838289 27635 heartbeater.cc:503] Master 127.9.123.126:40271 was elected leader, sending a full tablet report...
I20260926 15:07:25.838637 27916 log.cc:824] T 8accec55724f40e2b3bb1d841ca0de6a P 93508c72fd2d43379417cbf4d438a74b: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:25.843158 27915 raft_consensus.cc:348] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283297be26ff46d2a8a7601fc72676ac" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 44789 } } peers { permanent_uuid: "93508c72fd2d43379417cbf4d438a74b" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40789 } } peers { permanent_uuid: "27b9e914b7bc4d56b6a9ef41669e4d8e" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 38973 } }
I20260926 15:07:25.843729 27915 raft_consensus.cc:374] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:25.843889 27915 raft_consensus.cc:729] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283297be26ff46d2a8a7601fc72676ac, State: Initialized, Role: FOLLOWER
I20260926 15:07:25.844673 27915 consensus_queue.cc:260] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283297be26ff46d2a8a7601fc72676ac" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 44789 } } peers { permanent_uuid: "93508c72fd2d43379417cbf4d438a74b" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40789 } } peers { permanent_uuid: "27b9e914b7bc4d56b6a9ef41669e4d8e" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 38973 } }
I20260926 15:07:25.845909 27766 heartbeater.cc:503] Master 127.9.123.126:40271 was elected leader, sending a full tablet report...
I20260926 15:07:25.847142 27915 ts_tablet_manager.cc:1428] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260926 15:07:25.847785 27914 raft_consensus.cc:348] T 8accec55724f40e2b3bb1d841ca0de6a P 27b9e914b7bc4d56b6a9ef41669e4d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283297be26ff46d2a8a7601fc72676ac" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 44789 } } peers { permanent_uuid: "93508c72fd2d43379417cbf4d438a74b" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40789 } } peers { permanent_uuid: "27b9e914b7bc4d56b6a9ef41669e4d8e" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 38973 } }
I20260926 15:07:25.848244 27914 raft_consensus.cc:374] T 8accec55724f40e2b3bb1d841ca0de6a P 27b9e914b7bc4d56b6a9ef41669e4d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:25.848351 27914 raft_consensus.cc:729] T 8accec55724f40e2b3bb1d841ca0de6a P 27b9e914b7bc4d56b6a9ef41669e4d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27b9e914b7bc4d56b6a9ef41669e4d8e, State: Initialized, Role: FOLLOWER
I20260926 15:07:25.849076 27914 consensus_queue.cc:260] T 8accec55724f40e2b3bb1d841ca0de6a P 27b9e914b7bc4d56b6a9ef41669e4d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283297be26ff46d2a8a7601fc72676ac" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 44789 } } peers { permanent_uuid: "93508c72fd2d43379417cbf4d438a74b" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40789 } } peers { permanent_uuid: "27b9e914b7bc4d56b6a9ef41669e4d8e" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 38973 } }
I20260926 15:07:25.849944 27916 tablet_bootstrap.cc:492] T 8accec55724f40e2b3bb1d841ca0de6a P 93508c72fd2d43379417cbf4d438a74b: No bootstrap required, opened a new log
I20260926 15:07:25.850340 27916 ts_tablet_manager.cc:1397] T 8accec55724f40e2b3bb1d841ca0de6a P 93508c72fd2d43379417cbf4d438a74b: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.009s
I20260926 15:07:25.851140 27914 ts_tablet_manager.cc:1428] T 8accec55724f40e2b3bb1d841ca0de6a P 27b9e914b7bc4d56b6a9ef41669e4d8e: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260926 15:07:25.857898 27916 raft_consensus.cc:348] T 8accec55724f40e2b3bb1d841ca0de6a P 93508c72fd2d43379417cbf4d438a74b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283297be26ff46d2a8a7601fc72676ac" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 44789 } } peers { permanent_uuid: "93508c72fd2d43379417cbf4d438a74b" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40789 } } peers { permanent_uuid: "27b9e914b7bc4d56b6a9ef41669e4d8e" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 38973 } }
I20260926 15:07:25.858320 27916 raft_consensus.cc:374] T 8accec55724f40e2b3bb1d841ca0de6a P 93508c72fd2d43379417cbf4d438a74b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:25.858428 27916 raft_consensus.cc:729] T 8accec55724f40e2b3bb1d841ca0de6a P 93508c72fd2d43379417cbf4d438a74b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93508c72fd2d43379417cbf4d438a74b, State: Initialized, Role: FOLLOWER
I20260926 15:07:25.859066 27916 consensus_queue.cc:260] T 8accec55724f40e2b3bb1d841ca0de6a P 93508c72fd2d43379417cbf4d438a74b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283297be26ff46d2a8a7601fc72676ac" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 44789 } } peers { permanent_uuid: "93508c72fd2d43379417cbf4d438a74b" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40789 } } peers { permanent_uuid: "27b9e914b7bc4d56b6a9ef41669e4d8e" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 38973 } }
I20260926 15:07:25.860188 27896 heartbeater.cc:503] Master 127.9.123.126:40271 was elected leader, sending a full tablet report...
I20260926 15:07:25.860846 27916 ts_tablet_manager.cc:1428] T 8accec55724f40e2b3bb1d841ca0de6a P 93508c72fd2d43379417cbf4d438a74b: Time spent starting tablet: real 0.010s user 0.008s sys 0.001s
I20260926 15:07:25.883986 27920 raft_consensus.cc:482] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:07:25.884332 27920 raft_consensus.cc:504] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283297be26ff46d2a8a7601fc72676ac" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 44789 } } peers { permanent_uuid: "93508c72fd2d43379417cbf4d438a74b" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40789 } } peers { permanent_uuid: "27b9e914b7bc4d56b6a9ef41669e4d8e" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 38973 } }
I20260926 15:07:25.886111 27920 leader_election.cc:288] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93508c72fd2d43379417cbf4d438a74b (127.9.123.67:40789), 27b9e914b7bc4d56b6a9ef41669e4d8e (127.9.123.65:38973)
I20260926 15:07:25.893381 27911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:25.886620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:49041 (local address 127.9.123.67:40789)
0926 15:07:25.886816 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:25.886821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:25.887319 (+ 498us) server_negotiation.cc:407] Connection header received
0926 15:07:25.887533 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:25.887539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:25.887618 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:25.887723 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:25.888370 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.889361 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.891459 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.892441 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.892572 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:25.892621 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:25.893008 (+ 387us) server_negotiation.cc:299] Negotiation successful
0926 15:07:25.893137 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 15:07:25.893462 27923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:25.886519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:40789 (local address 127.9.123.66:49041)
0926 15:07:25.887191 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:25.887216 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:25.887356 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:25.887771 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:25.887776 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:25.887803 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:25.888213 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:25.888226 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.889492 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.889498 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:25.891249 (+ 1751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:25.891263 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.892644 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.892652 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:25.892830 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:25.892865 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 15:07:25.892969 (+ 104us) client_negotiation.cc:240] Negotiation successful
0926 15:07:25.893132 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":350,"threads_started":1}
I20260926 15:07:25.894073 27913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:25.887616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:43791 (local address 127.9.123.65:38973)
0926 15:07:25.887816 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:25.887820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:25.888173 (+ 353us) server_negotiation.cc:407] Connection header received
0926 15:07:25.888308 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:25.888314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:25.888417 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:25.888483 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:25.889165 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.890121 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.892093 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.893087 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.893272 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:25.893335 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:25.893703 (+ 368us) server_negotiation.cc:299] Negotiation successful
0926 15:07:25.893830 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 15:07:25.894105 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:25.887311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:38973 (local address 127.9.123.66:43791)
0926 15:07:25.888073 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:25.888086 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:25.888191 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:25.888539 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:25.888543 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:25.888570 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:25.888955 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:25.888968 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.890278 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.890283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:25.891879 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:25.891894 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.893306 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.893311 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:25.893489 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:25.893521 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 15:07:25.893652 (+ 131us) client_negotiation.cc:240] Negotiation successful
0926 15:07:25.893779 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"mutex_wait_us":1,"thread_start_us":301,"threads_started":1}
I20260926 15:07:25.894642 27851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8accec55724f40e2b3bb1d841ca0de6a" candidate_uuid: "283297be26ff46d2a8a7601fc72676ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93508c72fd2d43379417cbf4d438a74b" is_pre_election: true
I20260926 15:07:25.895108 27590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8accec55724f40e2b3bb1d841ca0de6a" candidate_uuid: "283297be26ff46d2a8a7601fc72676ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27b9e914b7bc4d56b6a9ef41669e4d8e" is_pre_election: true
I20260926 15:07:25.895228 27851 raft_consensus.cc:2454] T 8accec55724f40e2b3bb1d841ca0de6a P 93508c72fd2d43379417cbf4d438a74b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 283297be26ff46d2a8a7601fc72676ac in term 0.
I20260926 15:07:25.895596 27590 raft_consensus.cc:2454] T 8accec55724f40e2b3bb1d841ca0de6a P 27b9e914b7bc4d56b6a9ef41669e4d8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 283297be26ff46d2a8a7601fc72676ac in term 0.
I20260926 15:07:25.895884 27657 leader_election.cc:302] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 283297be26ff46d2a8a7601fc72676ac, 93508c72fd2d43379417cbf4d438a74b; no voters:
I20260926 15:07:25.896246 27920 raft_consensus.cc:2781] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:07:25.896369 27920 raft_consensus.cc:482] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:07:25.896469 27920 raft_consensus.cc:3037] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:25.898739 27920 raft_consensus.cc:504] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283297be26ff46d2a8a7601fc72676ac" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 44789 } } peers { permanent_uuid: "93508c72fd2d43379417cbf4d438a74b" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40789 } } peers { permanent_uuid: "27b9e914b7bc4d56b6a9ef41669e4d8e" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 38973 } }
I20260926 15:07:25.899302 27920 leader_election.cc:288] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac [CANDIDATE]: Term 1 election: Requested vote from peers 93508c72fd2d43379417cbf4d438a74b (127.9.123.67:40789), 27b9e914b7bc4d56b6a9ef41669e4d8e (127.9.123.65:38973)
I20260926 15:07:25.899843 27851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8accec55724f40e2b3bb1d841ca0de6a" candidate_uuid: "283297be26ff46d2a8a7601fc72676ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93508c72fd2d43379417cbf4d438a74b"
I20260926 15:07:25.899896 27590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8accec55724f40e2b3bb1d841ca0de6a" candidate_uuid: "283297be26ff46d2a8a7601fc72676ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27b9e914b7bc4d56b6a9ef41669e4d8e"
I20260926 15:07:25.900074 27851 raft_consensus.cc:3037] T 8accec55724f40e2b3bb1d841ca0de6a P 93508c72fd2d43379417cbf4d438a74b [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:25.900131 27590 raft_consensus.cc:3037] T 8accec55724f40e2b3bb1d841ca0de6a P 27b9e914b7bc4d56b6a9ef41669e4d8e [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:25.903287 27851 raft_consensus.cc:2454] T 8accec55724f40e2b3bb1d841ca0de6a P 93508c72fd2d43379417cbf4d438a74b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 283297be26ff46d2a8a7601fc72676ac in term 1.
I20260926 15:07:25.903287 27590 raft_consensus.cc:2454] T 8accec55724f40e2b3bb1d841ca0de6a P 27b9e914b7bc4d56b6a9ef41669e4d8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 283297be26ff46d2a8a7601fc72676ac in term 1.
I20260926 15:07:25.903736 27657 leader_election.cc:302] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27b9e914b7bc4d56b6a9ef41669e4d8e, 283297be26ff46d2a8a7601fc72676ac; no voters:
I20260926 15:07:25.904031 27920 raft_consensus.cc:2781] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:07:25.904459 27920 raft_consensus.cc:686] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac [term 1 LEADER]: Becoming Leader. State: Replica: 283297be26ff46d2a8a7601fc72676ac, State: Running, Role: LEADER
I20260926 15:07:25.904943 27920 consensus_queue.cc:237] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283297be26ff46d2a8a7601fc72676ac" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 44789 } } peers { permanent_uuid: "93508c72fd2d43379417cbf4d438a74b" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40789 } } peers { permanent_uuid: "27b9e914b7bc4d56b6a9ef41669e4d8e" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 38973 } }
I20260926 15:07:25.909989 27449 catalog_manager.cc:5495] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 283297be26ff46d2a8a7601fc72676ac (127.9.123.66). New cstate: current_term: 1 leader_uuid: "283297be26ff46d2a8a7601fc72676ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "283297be26ff46d2a8a7601fc72676ac" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 44789 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93508c72fd2d43379417cbf4d438a74b" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27b9e914b7bc4d56b6a9ef41669e4d8e" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 38973 } health_report { overall_health: UNKNOWN } } }
I20260926 15:07:25.931210 27910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:25.925939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34246 (local address 127.9.123.66:44789)
0926 15:07:25.926150 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:25.926155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:25.926174 (+ 19us) server_negotiation.cc:407] Connection header received
0926 15:07:25.926352 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:25.926355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:25.926451 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:25.926590 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:07:25.926884 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.927581 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.929000 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:25.929489 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:25.929592 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:25.930104 (+ 512us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:07:25.930658 (+ 554us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:07:25.930765 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:25.930879 (+ 114us) server_negotiation.cc:299] Negotiation successful
0926 15:07:25.930947 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 15:07:25.939729 27590 raft_consensus.cc:1261] T 8accec55724f40e2b3bb1d841ca0de6a P 27b9e914b7bc4d56b6a9ef41669e4d8e [term 1 FOLLOWER]: Refusing update from remote peer 283297be26ff46d2a8a7601fc72676ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:07:25.939729 27851 raft_consensus.cc:1261] T 8accec55724f40e2b3bb1d841ca0de6a P 93508c72fd2d43379417cbf4d438a74b [term 1 FOLLOWER]: Refusing update from remote peer 283297be26ff46d2a8a7601fc72676ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:07:25.940734 27920 consensus_queue.cc:1035] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "93508c72fd2d43379417cbf4d438a74b" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 40789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:07:25.941076 27925 consensus_queue.cc:1035] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "27b9e914b7bc4d56b6a9ef41669e4d8e" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 38973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:07:25.952858 27928 mvcc.cc:205] Tried to move back new op lower bound from 7333622767358226432 to 7333622767231008768. Current Snapshot: MvccSnapshot[applied={T|T < 7333622767358226432}]
I20260926 15:07:25.968135 27449 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:07:25.968416 27449 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:07:25.972585 27449 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5d658ff58a23463ab9c62bac5ab9615d: Sending DeleteTablet for 3 replicas of tablet 8accec55724f40e2b3bb1d841ca0de6a
I20260926 15:07:25.974030 27570 tablet_service.cc:1514] Processing DeleteTablet for tablet 8accec55724f40e2b3bb1d841ca0de6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:07:25 UTC) from {username='slave'} at 127.0.0.1:38922
I20260926 15:07:25.974030 27701 tablet_service.cc:1514] Processing DeleteTablet for tablet 8accec55724f40e2b3bb1d841ca0de6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:07:25 UTC) from {username='slave'} at 127.0.0.1:34230
I20260926 15:07:25.974043 27831 tablet_service.cc:1514] Processing DeleteTablet for tablet 8accec55724f40e2b3bb1d841ca0de6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:07:25 UTC) from {username='slave'} at 127.0.0.1:40740
I20260926 15:07:25.974790 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 27508
I20260926 15:07:25.985553 27939 tablet_replica.cc:330] T 8accec55724f40e2b3bb1d841ca0de6a P 27b9e914b7bc4d56b6a9ef41669e4d8e: stopping tablet replica
I20260926 15:07:25.986943 27939 raft_consensus.cc:2229] T 8accec55724f40e2b3bb1d841ca0de6a P 27b9e914b7bc4d56b6a9ef41669e4d8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:07:25.988286 27939 raft_consensus.cc:2258] T 8accec55724f40e2b3bb1d841ca0de6a P 27b9e914b7bc4d56b6a9ef41669e4d8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:07:25.993278 27939 ts_tablet_manager.cc:1905] T 8accec55724f40e2b3bb1d841ca0de6a P 27b9e914b7bc4d56b6a9ef41669e4d8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:07:26.002221 27938 tablet_replica.cc:330] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac: stopping tablet replica
I20260926 15:07:26.002875 27940 tablet_replica.cc:330] T 8accec55724f40e2b3bb1d841ca0de6a P 93508c72fd2d43379417cbf4d438a74b: stopping tablet replica
I20260926 15:07:26.003369 27940 raft_consensus.cc:2229] T 8accec55724f40e2b3bb1d841ca0de6a P 93508c72fd2d43379417cbf4d438a74b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:07:26.003937 27940 raft_consensus.cc:2258] T 8accec55724f40e2b3bb1d841ca0de6a P 93508c72fd2d43379417cbf4d438a74b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:07:26.004765 27938 raft_consensus.cc:2229] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:07:26.005539 27938 raft_consensus.cc:2258] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:07:26.005753 27913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:25.976125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38938 (local address 127.9.123.65:38973)
0926 15:07:25.976310 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:25.976324 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:25.997682 (+ 21358us) server_negotiation.cc:407] Connection header received
0926 15:07:25.997864 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:25.997870 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:25.997999 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:25.998117 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:25.998830 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:26.000176 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:26.002386 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:26.002911 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:26.003019 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:26.004251 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:26.004262 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:26.004280 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:26.004921 (+ 641us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:26.004988 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:26.005014 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:26.005164 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:26.005283 (+ 119us) server_negotiation.cc:299] Negotiation successful
0926 15:07:26.005403 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 15:07:26.006690 27630 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:26.008328 27940 ts_tablet_manager.cc:1905] T 8accec55724f40e2b3bb1d841ca0de6a P 93508c72fd2d43379417cbf4d438a74b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:07:26.008963 27938 ts_tablet_manager.cc:1905] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:07:26.017086 27938 ts_tablet_manager.cc:1918] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:07:26.017422 27938 log.cc:1196] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/8accec55724f40e2b3bb1d841ca0de6a
I20260926 15:07:26.018111 27938 ts_tablet_manager.cc:1939] T 8accec55724f40e2b3bb1d841ca0de6a P 283297be26ff46d2a8a7601fc72676ac: Deleting consensus metadata
I20260926 15:07:26.018630 27940 ts_tablet_manager.cc:1918] T 8accec55724f40e2b3bb1d841ca0de6a P 93508c72fd2d43379417cbf4d438a74b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:07:26.018873 27940 log.cc:1196] T 8accec55724f40e2b3bb1d841ca0de6a P 93508c72fd2d43379417cbf4d438a74b: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/8accec55724f40e2b3bb1d841ca0de6a
I20260926 15:07:26.019447 27940 ts_tablet_manager.cc:1939] T 8accec55724f40e2b3bb1d841ca0de6a P 93508c72fd2d43379417cbf4d438a74b: Deleting consensus metadata
I20260926 15:07:26.020212 27433 catalog_manager.cc:4841] TS 283297be26ff46d2a8a7601fc72676ac (127.9.123.66:44789): tablet 8accec55724f40e2b3bb1d841ca0de6a (table test-table [id=4dbcc285d2dc451282fe15ef066f1c54]) successfully deleted
I20260926 15:07:26.021229 27436 catalog_manager.cc:4841] TS 93508c72fd2d43379417cbf4d438a74b (127.9.123.67:40789): tablet 8accec55724f40e2b3bb1d841ca0de6a (table test-table [id=4dbcc285d2dc451282fe15ef066f1c54]) successfully deleted
I20260926 15:07:26.131937 27939 ts_tablet_manager.cc:1918] T 8accec55724f40e2b3bb1d841ca0de6a P 27b9e914b7bc4d56b6a9ef41669e4d8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:07:26.132208 27939 log.cc:1196] T 8accec55724f40e2b3bb1d841ca0de6a P 27b9e914b7bc4d56b6a9ef41669e4d8e: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790434939095847-9709-0/minicluster-data/ts-0/wal/wals/8accec55724f40e2b3bb1d841ca0de6a
I20260926 15:07:26.132663 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 27508
I20260926 15:07:26.132714 27939 ts_tablet_manager.cc:1939] T 8accec55724f40e2b3bb1d841ca0de6a P 27b9e914b7bc4d56b6a9ef41669e4d8e: Deleting consensus metadata
W20260926 15:07:26.153263 27436 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.123.65:38973 (error 108)
I20260926 15:07:26.153348 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 27639
I20260926 15:07:26.155288 27912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:26.154773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:38973 (local address 127.0.0.1:38952)
0926 15:07:26.154975 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:26.155106 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:38973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260926 15:07:26.155625 27436 catalog_manager.cc:4584] TS 27b9e914b7bc4d56b6a9ef41669e4d8e (127.9.123.65:38973): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8accec55724f40e2b3bb1d841ca0de6a: Network error: Client connection negotiation failed: client connection to 127.9.123.65:38973: connect: Connection refused (error 111)
I20260926 15:07:26.162683 27910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:26.154782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34252 (local address 127.9.123.66:44789)
0926 15:07:26.154962 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:26.154967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:26.154986 (+ 19us) server_negotiation.cc:407] Connection header received
0926 15:07:26.155056 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:26.155061 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:26.155168 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:26.155270 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:26.155988 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:26.156827 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:26.158874 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:26.159376 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:26.159482 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:26.160785 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:26.160798 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:26.160817 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:26.161669 (+ 852us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:26.161731 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:26.161763 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:26.161873 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:26.162007 (+ 134us) server_negotiation.cc:299] Negotiation successful
0926 15:07:26.162120 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 15:07:26.163655 27761 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:26.209643 27912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:26.209065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:38973 (local address 127.0.0.1:38962)
0926 15:07:26.209328 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:26.209438 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:38973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260926 15:07:26.210806 27912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:26.210371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:38973 (local address 127.0.0.1:38970)
0926 15:07:26.210540 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:26.210630 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:38973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260926 15:07:26.271466 27912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:26.270928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:38973 (local address 127.0.0.1:38974)
0926 15:07:26.271166 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:26.271265 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:38973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260926 15:07:26.272710 27912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:26.272293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:38973 (local address 127.0.0.1:38984)
0926 15:07:26.272474 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:26.272558 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:38973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260926 15:07:26.279363 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 27639
I20260926 15:07:26.294950 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 27769
I20260926 15:07:26.302281 27911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:26.295686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40750 (local address 127.9.123.67:40789)
0926 15:07:26.295842 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:26.295846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:26.295916 (+ 70us) server_negotiation.cc:407] Connection header received
0926 15:07:26.296136 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:26.296141 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:26.296236 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:26.296344 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:26.296800 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:26.297627 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:26.299059 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:26.299463 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:26.299568 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:26.300759 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:26.300768 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:26.300785 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:26.301620 (+ 835us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:26.301673 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:26.301696 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:26.301796 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:26.301915 (+ 119us) server_negotiation.cc:299] Negotiation successful
0926 15:07:26.302039 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 15:07:26.302918 27891 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:26.354072 27912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:26.353562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:38973 (local address 127.0.0.1:38994)
0926 15:07:26.353762 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:26.353867 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:38973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260926 15:07:26.355167 27912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:26.354758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:38973 (local address 127.0.0.1:39004)
0926 15:07:26.354953 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:26.355051 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:38973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260926 15:07:26.396341 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 27769
I20260926 15:07:26.409334 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 27418
I20260926 15:07:26.410490 27479 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:26.518790 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 27418
2026-09-26T15:07:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2809 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:07:26 dist-test-slave-7153 krb5kdc[27949](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:07:26 dist-test-slave-7153 krb5kdc[27949](info): set up 2 sockets
Sep 26 15:07:26 dist-test-slave-7153 krb5kdc[27949](info): commencing operation
krb5kdc: starting...
W20260926 15:07:27.541083 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.002s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:07:27 dist-test-slave-7153 krb5kdc[27949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:07:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:07:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:07:27.781132 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:39333
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:42331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:39333
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:07:28.110239 27962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:28.110538 27962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:28.110627 27962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:28.118043 27962 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:07:28.118167 27962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:28.118216 27962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:28.118271 27962 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:07:28.118314 27962 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:07:28.128511 27962 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:42331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:39333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:39333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:28.129855 27962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:28.131083 27962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:28.142573 27970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:28.142544 27968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:28.142599 27967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:28.143420 27962 server_base.cc:1034] running on GCE node
I20260926 15:07:28.144186 27962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:28.145714 27962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:28.147027 27962 hybrid_clock.cc:648] HybridClock initialized: now 1790435248146966 us; error 46 us; skew 500 ppm
I20260926 15:07:28.147437 27962 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:28 dist-test-slave-7153 krb5kdc[27949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435248, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:28.150779 27962 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:07:28.152534 27962 webserver.cc:460] Webserver started at http://127.9.123.126:46247/ using document root <none> and password file <none>
I20260926 15:07:28.153330 27962 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:28.153455 27962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:28.153743 27962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:28.156816 27962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "a093ff918cb347ab92bd43f77afb3b7b"
format_stamp: "Formatted at 2026-09-26 15:07:28 on dist-test-slave-7153"
server_key: "bb2cce4fe5d14d343c6d1118221e3f10"
server_key_iv: "0f7845ba447c3274085c6074f12421b5"
server_key_version: "encryptionkey@0"
I20260926 15:07:28.157635 27962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "a093ff918cb347ab92bd43f77afb3b7b"
format_stamp: "Formatted at 2026-09-26 15:07:28 on dist-test-slave-7153"
server_key: "bb2cce4fe5d14d343c6d1118221e3f10"
server_key_iv: "0f7845ba447c3274085c6074f12421b5"
server_key_version: "encryptionkey@0"
I20260926 15:07:28.162835 27962 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 15:07:28.166412 27977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:28.167773 27962 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:07:28.167965 27962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "a093ff918cb347ab92bd43f77afb3b7b"
format_stamp: "Formatted at 2026-09-26 15:07:28 on dist-test-slave-7153"
server_key: "bb2cce4fe5d14d343c6d1118221e3f10"
server_key_iv: "0f7845ba447c3274085c6074f12421b5"
server_key_version: "encryptionkey@0"
I20260926 15:07:28.168154 27962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:28.190816 27962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:28.195225 27962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:28.195501 27962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:28.213466 27962 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:39333
I20260926 15:07:28.213478 28028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:39333 every 8 connection(s)
I20260926 15:07:28.214813 27962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:07:28.218505 28029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:28.224609 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 27962
I20260926 15:07:28.224834 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:07:28.225412 9709 external_mini_cluster.cc:1428] Setting key 9106e465cffb671e16473b320834153a
I20260926 15:07:28.230690 28029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a093ff918cb347ab92bd43f77afb3b7b: Bootstrap starting.
I20260926 15:07:28.233940 28029 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a093ff918cb347ab92bd43f77afb3b7b: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:28.235015 28029 log.cc:824] T 00000000000000000000000000000000 P a093ff918cb347ab92bd43f77afb3b7b: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:28.238193 28029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a093ff918cb347ab92bd43f77afb3b7b: No bootstrap required, opened a new log
Sep 26 15:07:28 dist-test-slave-7153 krb5kdc[27949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:07:28.248227 28029 raft_consensus.cc:348] T 00000000000000000000000000000000 P a093ff918cb347ab92bd43f77afb3b7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a093ff918cb347ab92bd43f77afb3b7b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39333 } }
I20260926 15:07:28.248764 28029 raft_consensus.cc:374] T 00000000000000000000000000000000 P a093ff918cb347ab92bd43f77afb3b7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:28.248893 28029 raft_consensus.cc:729] T 00000000000000000000000000000000 P a093ff918cb347ab92bd43f77afb3b7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a093ff918cb347ab92bd43f77afb3b7b, State: Initialized, Role: FOLLOWER
I20260926 15:07:28.249643 28029 consensus_queue.cc:260] T 00000000000000000000000000000000 P a093ff918cb347ab92bd43f77afb3b7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a093ff918cb347ab92bd43f77afb3b7b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39333 } }
I20260926 15:07:28.249984 28029 raft_consensus.cc:388] T 00000000000000000000000000000000 P a093ff918cb347ab92bd43f77afb3b7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:07:28.250130 28029 raft_consensus.cc:482] T 00000000000000000000000000000000 P a093ff918cb347ab92bd43f77afb3b7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:07:28.250325 28029 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a093ff918cb347ab92bd43f77afb3b7b [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:28.250824 28032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:28.227544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34562 (local address 127.9.123.126:39333)
0926 15:07:28.228457 (+ 913us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:28.228470 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:28.228517 (+ 47us) server_negotiation.cc:407] Connection header received
0926 15:07:28.230409 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:28.230436 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:28.230910 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:28.231323 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:28.232845 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:28.234261 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:28.235700 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:28.236538 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:28.236740 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:28.241017 (+ 4277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:28.241031 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:28.241054 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:28.241132 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:28.245493 (+ 4361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:28.246382 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:28.246397 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:28.246407 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:28.246679 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:28.247187 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:28.247193 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:28.247196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:28.247998 (+ 802us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:28.248314 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:28.248745 (+ 431us) server_negotiation.cc:299] Negotiation successful
0926 15:07:28.249299 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":422,"threads_started":1}
I20260926 15:07:28.252954 28029 raft_consensus.cc:504] T 00000000000000000000000000000000 P a093ff918cb347ab92bd43f77afb3b7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a093ff918cb347ab92bd43f77afb3b7b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39333 } }
I20260926 15:07:28.253561 28029 leader_election.cc:302] T 00000000000000000000000000000000 P a093ff918cb347ab92bd43f77afb3b7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a093ff918cb347ab92bd43f77afb3b7b; no voters:
I20260926 15:07:28.254323 28029 leader_election.cc:288] T 00000000000000000000000000000000 P a093ff918cb347ab92bd43f77afb3b7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:07:28.254400 28034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a093ff918cb347ab92bd43f77afb3b7b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:07:28.255342 28034 raft_consensus.cc:686] T 00000000000000000000000000000000 P a093ff918cb347ab92bd43f77afb3b7b [term 1 LEADER]: Becoming Leader. State: Replica: a093ff918cb347ab92bd43f77afb3b7b, State: Running, Role: LEADER
I20260926 15:07:28.255968 28034 consensus_queue.cc:237] T 00000000000000000000000000000000 P a093ff918cb347ab92bd43f77afb3b7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a093ff918cb347ab92bd43f77afb3b7b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39333 } }
I20260926 15:07:28.257352 28029 sys_catalog.cc:564] T 00000000000000000000000000000000 P a093ff918cb347ab92bd43f77afb3b7b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:07:28.259646 28036 sys_catalog.cc:455] T 00000000000000000000000000000000 P a093ff918cb347ab92bd43f77afb3b7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a093ff918cb347ab92bd43f77afb3b7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a093ff918cb347ab92bd43f77afb3b7b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39333 } } }
I20260926 15:07:28.260061 28035 sys_catalog.cc:455] T 00000000000000000000000000000000 P a093ff918cb347ab92bd43f77afb3b7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a093ff918cb347ab92bd43f77afb3b7b. Latest consensus state: current_term: 1 leader_uuid: "a093ff918cb347ab92bd43f77afb3b7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a093ff918cb347ab92bd43f77afb3b7b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39333 } } }
I20260926 15:07:28.260761 28036 sys_catalog.cc:458] T 00000000000000000000000000000000 P a093ff918cb347ab92bd43f77afb3b7b [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:28.260876 28035 sys_catalog.cc:458] T 00000000000000000000000000000000 P a093ff918cb347ab92bd43f77afb3b7b [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:28.262145 28040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:07:28.267642 28040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:07:28.278817 28040 catalog_manager.cc:1347] Generated new cluster ID: c0e6173885f74af2ab70ad303c582268
I20260926 15:07:28.278966 28040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:07:28.291906 28040 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:07:28.293339 28040 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:07:28.305621 28040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a093ff918cb347ab92bd43f77afb3b7b: Generated new TSK 0
I20260926 15:07:28.306511 28040 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.123.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
I20260926 15:07:28.376607 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:39333
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:42331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:07:28.710642 28055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:28.710978 28055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:28.711112 28055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:28.718545 28055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:28.718672 28055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:28.718812 28055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:07:28.728718 28055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:42331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:39333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:28.730053 28055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:28.731441 28055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:28.745296 28061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:28.745337 28060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:28.745483 28063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:28.746229 28055 server_base.cc:1034] running on GCE node
I20260926 15:07:28.746829 28055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:28.747797 28055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:28.749035 28055 hybrid_clock.cc:648] HybridClock initialized: now 1790435248748967 us; error 58 us; skew 500 ppm
I20260926 15:07:28.749387 28055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:28 dist-test-slave-7153 krb5kdc[27949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435248, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:28.752593 28055 init.cc:377] Logged in from keytab as kudu/127.9.123.65@KRBTEST.COM (short username kudu)
I20260926 15:07:28.754200 28055 webserver.cc:460] Webserver started at http://127.9.123.65:39129/ using document root <none> and password file <none>
I20260926 15:07:28.754904 28055 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:28.755023 28055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:28.755329 28055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:28.758437 28055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "b4bd55c865684e2d9fc8b64a25d0d94e"
format_stamp: "Formatted at 2026-09-26 15:07:28 on dist-test-slave-7153"
server_key: "3ef243bf5bad650acd8f78d21b62ccb6"
server_key_iv: "53ae7a83215254816c1b3d31945a9a9b"
server_key_version: "encryptionkey@0"
I20260926 15:07:28.759241 28055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "b4bd55c865684e2d9fc8b64a25d0d94e"
format_stamp: "Formatted at 2026-09-26 15:07:28 on dist-test-slave-7153"
server_key: "3ef243bf5bad650acd8f78d21b62ccb6"
server_key_iv: "53ae7a83215254816c1b3d31945a9a9b"
server_key_version: "encryptionkey@0"
I20260926 15:07:28.764575 28055 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 15:07:28.768406 28070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:28.769923 28055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:07:28.770087 28055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "b4bd55c865684e2d9fc8b64a25d0d94e"
format_stamp: "Formatted at 2026-09-26 15:07:28 on dist-test-slave-7153"
server_key: "3ef243bf5bad650acd8f78d21b62ccb6"
server_key_iv: "53ae7a83215254816c1b3d31945a9a9b"
server_key_version: "encryptionkey@0"
I20260926 15:07:28.770299 28055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:28.792196 28055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:28.796692 28055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:28.796948 28055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:28.797958 28055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:07:28.799271 28055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:07:28.799345 28055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:28.799425 28055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:07:28.799467 28055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:28.828685 28055 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:38305
I20260926 15:07:28.828739 28182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:38305 every 8 connection(s)
I20260926 15:07:28.830045 28055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:07:28.834174 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 28055
I20260926 15:07:28.834352 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:07:28.834784 9709 external_mini_cluster.cc:1428] Setting key 14d8699571874f20e7a552f83148e69c
Sep 26 15:07:28 dist-test-slave-7153 krb5kdc[27949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435248, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:07:28.853210 28185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:28.832639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:47879 (local address 127.9.123.126:39333)
0926 15:07:28.833119 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:28.833123 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:28.834194 (+ 1071us) server_negotiation.cc:407] Connection header received
0926 15:07:28.836235 (+ 2041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:28.836240 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:28.836325 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:28.836441 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:28.838569 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:28.839323 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:28.841432 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:28.841878 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:28.841985 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:28.846164 (+ 4179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:28.846172 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:28.846177 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:28.846247 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:28.849585 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:28.850548 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:28.850553 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:28.850557 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:28.850710 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:28.851222 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:28.851227 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:28.851230 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:28.851578 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:28.851715 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:28.852554 (+ 839us) server_negotiation.cc:299] Negotiation successful
0926 15:07:28.852787 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":226,"threads_started":1}
I20260926 15:07:28.854144 28186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:28.833235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39333 (local address 127.9.123.65:47879)
0926 15:07:28.834020 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:28.834067 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:28.835951 (+ 1884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:28.836688 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:28.836700 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:28.837533 (+ 833us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:07:28.838355 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:28.838382 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:28.839498 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:28.839503 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:28.841280 (+ 1777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:28.841296 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:28.842107 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:28.842113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:28.842418 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:28.843115 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:07:28.843146 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:07:28.845839 (+ 2693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:07:28.849871 (+ 4032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:07:28.849880 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:07:28.849895 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:07:28.850401 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:07:28.850862 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:07:28.850866 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:07:28.850868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:07:28.851099 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:07:28.851750 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:07:28.851757 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:07:28.852140 (+ 383us) client_negotiation.cc:769] Sending connection context
0926 15:07:28.852402 (+ 262us) client_negotiation.cc:240] Negotiation successful
0926 15:07:28.852858 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":286,"threads_started":1}
I20260926 15:07:28.855794 28183 heartbeater.cc:347] Connected to a master server at 127.9.123.126:39333
I20260926 15:07:28.856182 28183 heartbeater.cc:464] Registering TS with master...
I20260926 15:07:28.857147 28183 heartbeater.cc:511] Master 127.9.123.126:39333 requested a full tablet report, sending...
I20260926 15:07:28.859665 27994 ts_manager.cc:194] Registered new tserver with Master: b4bd55c865684e2d9fc8b64a25d0d94e (127.9.123.65:38305)
WARNING: no policy specified for kudu/127.9.123.66@KRBTEST.COM; defaulting to no policy
I20260926 15:07:28.861346 27994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.65@KRBTEST.COM'} at 127.9.123.65:47879
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
I20260926 15:07:28.893442 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:39333
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:42331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:07:29.224613 28189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:29.224967 28189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:29.225137 28189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:29.232640 28189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:29.232788 28189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:29.232952 28189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:07:29.243597 28189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:42331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:39333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:29.244984 28189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:29.246388 28189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:29.259827 28194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:29.259770 28197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:29.259757 28195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:29.260478 28189 server_base.cc:1034] running on GCE node
I20260926 15:07:29.261197 28189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:29.262208 28189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:29.263411 28189 hybrid_clock.cc:648] HybridClock initialized: now 1790435249263361 us; error 47 us; skew 500 ppm
I20260926 15:07:29.263723 28189 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:29 dist-test-slave-7153 krb5kdc[27949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435249, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:29.271220 28189 init.cc:377] Logged in from keytab as kudu/127.9.123.66@KRBTEST.COM (short username kudu)
I20260926 15:07:29.272903 28189 webserver.cc:460] Webserver started at http://127.9.123.66:44569/ using document root <none> and password file <none>
I20260926 15:07:29.273715 28189 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:29.273840 28189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:29.274155 28189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:29.277269 28189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "515b40ff81f442478f09bfb70de8ee19"
format_stamp: "Formatted at 2026-09-26 15:07:29 on dist-test-slave-7153"
server_key: "6ec361ab4ec5f974f0b73241a884ea1b"
server_key_iv: "9f44698ca42985d37483ba48f2c56821"
server_key_version: "encryptionkey@0"
I20260926 15:07:29.278038 28189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "515b40ff81f442478f09bfb70de8ee19"
format_stamp: "Formatted at 2026-09-26 15:07:29 on dist-test-slave-7153"
server_key: "6ec361ab4ec5f974f0b73241a884ea1b"
server_key_iv: "9f44698ca42985d37483ba48f2c56821"
server_key_version: "encryptionkey@0"
I20260926 15:07:29.283165 28189 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 15:07:29.286684 28204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:29.288002 28189 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 15:07:29.288153 28189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "515b40ff81f442478f09bfb70de8ee19"
format_stamp: "Formatted at 2026-09-26 15:07:29 on dist-test-slave-7153"
server_key: "6ec361ab4ec5f974f0b73241a884ea1b"
server_key_iv: "9f44698ca42985d37483ba48f2c56821"
server_key_version: "encryptionkey@0"
I20260926 15:07:29.288345 28189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:29.328128 28189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:29.332670 28189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:29.332969 28189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:29.334009 28189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:07:29.335465 28189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:07:29.335553 28189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:29.335675 28189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:07:29.335726 28189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:29.367120 28189 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:42949
I20260926 15:07:29.367202 28316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:42949 every 8 connection(s)
I20260926 15:07:29.368383 28189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:07:29.377856 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 28189
I20260926 15:07:29.378036 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:07:29.378480 9709 external_mini_cluster.cc:1428] Setting key 44e94b8164efd35eda9d186b82aec031
Sep 26 15:07:29 dist-test-slave-7153 krb5kdc[27949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435249, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:07:29.391455 28319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:29.370991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:43199 (local address 127.9.123.126:39333)
0926 15:07:29.371480 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:29.371487 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:29.372608 (+ 1121us) server_negotiation.cc:407] Connection header received
0926 15:07:29.374648 (+ 2040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:29.374653 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:29.374767 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:29.374900 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:29.376707 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:29.377580 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:29.379463 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:29.379958 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:29.380068 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:29.384194 (+ 4126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:29.384201 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:29.384206 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:29.384256 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:29.387726 (+ 3470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:29.388789 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:29.388796 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:29.388799 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:29.388960 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:29.389521 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:29.389526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:29.389529 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:29.389838 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:29.389981 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:29.390824 (+ 843us) server_negotiation.cc:299] Negotiation successful
0926 15:07:29.391080 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":243,"threads_started":1}
I20260926 15:07:29.392606 28320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:29.371617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39333 (local address 127.9.123.66:43199)
0926 15:07:29.372436 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:29.372485 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:29.374321 (+ 1836us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:29.375127 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:29.375139 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:29.375767 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:07:29.376441 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:29.376464 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:29.377809 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:29.377814 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:29.379310 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:29.379328 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:29.380300 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:29.380313 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:29.380584 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:29.381430 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:07:29.381457 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:07:29.383849 (+ 2392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:07:29.387958 (+ 4109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:07:29.387970 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:07:29.387991 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:07:29.388628 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:07:29.389135 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:07:29.389141 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:07:29.389146 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:07:29.389389 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:07:29.390002 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:07:29.390011 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:07:29.390398 (+ 387us) client_negotiation.cc:769] Sending connection context
0926 15:07:29.390659 (+ 261us) client_negotiation.cc:240] Negotiation successful
0926 15:07:29.391263 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":305,"threads_started":1}
I20260926 15:07:29.394323 28317 heartbeater.cc:347] Connected to a master server at 127.9.123.126:39333
I20260926 15:07:29.394682 28317 heartbeater.cc:464] Registering TS with master...
I20260926 15:07:29.395612 28317 heartbeater.cc:511] Master 127.9.123.126:39333 requested a full tablet report, sending...
I20260926 15:07:29.397307 27994 ts_manager.cc:194] Registered new tserver with Master: 515b40ff81f442478f09bfb70de8ee19 (127.9.123.66:42949)
I20260926 15:07:29.398406 27994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.66@KRBTEST.COM'} at 127.9.123.66:43199
WARNING: no policy specified for kudu/127.9.123.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
I20260926 15:07:29.432658 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:39333
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:42331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:07:29.764981 28323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:29.765327 28323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:29.765466 28323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:29.772789 28323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:29.772908 28323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:29.773062 28323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:07:29.783057 28323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:42331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:39333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:29.784335 28323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:29.785583 28323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:29.798043 28328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:29.798130 28329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:29.798529 28331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:29.798755 28323 server_base.cc:1034] running on GCE node
I20260926 15:07:29.799320 28323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:29.800213 28323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:29.801414 28323 hybrid_clock.cc:648] HybridClock initialized: now 1790435249801360 us; error 45 us; skew 500 ppm
I20260926 15:07:29.801745 28323 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:29 dist-test-slave-7153 krb5kdc[27949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435249, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:29.804775 28323 init.cc:377] Logged in from keytab as kudu/127.9.123.67@KRBTEST.COM (short username kudu)
I20260926 15:07:29.806270 28323 webserver.cc:460] Webserver started at http://127.9.123.67:38157/ using document root <none> and password file <none>
I20260926 15:07:29.806943 28323 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:29.807060 28323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:29.807358 28323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:29.810282 28323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "d64c17797a8d4bc2b31e6802da89f01d"
format_stamp: "Formatted at 2026-09-26 15:07:29 on dist-test-slave-7153"
server_key: "be26d637f31396f0c87c0ef3cd890b11"
server_key_iv: "c35b9a513ace1c7f9b30a992dc316a1c"
server_key_version: "encryptionkey@0"
I20260926 15:07:29.811041 28323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "d64c17797a8d4bc2b31e6802da89f01d"
format_stamp: "Formatted at 2026-09-26 15:07:29 on dist-test-slave-7153"
server_key: "be26d637f31396f0c87c0ef3cd890b11"
server_key_iv: "c35b9a513ace1c7f9b30a992dc316a1c"
server_key_version: "encryptionkey@0"
I20260926 15:07:29.816401 28323 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260926 15:07:29.820348 28338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:29.821856 28323 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260926 15:07:29.822017 28323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "d64c17797a8d4bc2b31e6802da89f01d"
format_stamp: "Formatted at 2026-09-26 15:07:29 on dist-test-slave-7153"
server_key: "be26d637f31396f0c87c0ef3cd890b11"
server_key_iv: "c35b9a513ace1c7f9b30a992dc316a1c"
server_key_version: "encryptionkey@0"
I20260926 15:07:29.822204 28323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:29.861544 28323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:29.864921 28183 heartbeater.cc:503] Master 127.9.123.126:39333 was elected leader, sending a full tablet report...
I20260926 15:07:29.866037 28323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:29.866314 28323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:29.867352 28323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:07:29.868633 28323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:07:29.868716 28323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:29.868836 28323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:07:29.868888 28323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:29.899187 28323 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:43021
I20260926 15:07:29.899292 28450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:43021 every 8 connection(s)
I20260926 15:07:29.900452 28323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:07:29.906540 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 28323
I20260926 15:07:29.906713 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:07:29.907163 9709 external_mini_cluster.cc:1428] Setting key 940cfc1dd939bcdae25624d9e7a3213b
Sep 26 15:07:29 dist-test-slave-7153 krb5kdc[27949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435249, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:07:29.920603 28453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:29.902937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:44587 (local address 127.9.123.126:39333)
0926 15:07:29.903362 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:29.903367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:29.904327 (+ 960us) server_negotiation.cc:407] Connection header received
0926 15:07:29.906237 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:29.906243 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:29.906352 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:29.906448 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:29.908194 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:29.909180 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:29.911035 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:29.911461 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:29.911567 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:29.915048 (+ 3481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:29.915053 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:29.915057 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:29.915095 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:29.917736 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:29.918487 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:29.918492 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:29.918494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:29.918636 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:29.919019 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:29.919022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:29.919024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:29.919298 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:29.919434 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:29.920135 (+ 701us) server_negotiation.cc:299] Negotiation successful
0926 15:07:29.920331 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":221,"threads_started":1}
I20260926 15:07:29.921859 28454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:29.903432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39333 (local address 127.9.123.67:44587)
0926 15:07:29.904145 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:29.904186 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:29.905946 (+ 1760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:29.906732 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:29.906742 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:29.907337 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:07:29.907970 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:29.907996 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:29.909332 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:29.909337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:29.910883 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:29.910896 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:29.911643 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:29.911647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:29.911872 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:29.912526 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:07:29.912562 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:07:29.914818 (+ 2256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:07:29.917890 (+ 3072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:07:29.917898 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:07:29.917914 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:07:29.918358 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:07:29.918744 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:07:29.918748 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:07:29.918751 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:07:29.918939 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:07:29.919400 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:07:29.919406 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:07:29.919759 (+ 353us) client_negotiation.cc:769] Sending connection context
0926 15:07:29.920001 (+ 242us) client_negotiation.cc:240] Negotiation successful
0926 15:07:29.920438 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":279,"threads_started":1}
I20260926 15:07:29.923237 28451 heartbeater.cc:347] Connected to a master server at 127.9.123.126:39333
I20260926 15:07:29.923549 28451 heartbeater.cc:464] Registering TS with master...
I20260926 15:07:29.924304 28451 heartbeater.cc:511] Master 127.9.123.126:39333 requested a full tablet report, sending...
I20260926 15:07:29.925781 27993 ts_manager.cc:194] Registered new tserver with Master: d64c17797a8d4bc2b31e6802da89f01d (127.9.123.67:43021)
I20260926 15:07:29.926677 27993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.67@KRBTEST.COM'} at 127.9.123.67:44587
I20260926 15:07:29.935292 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 15:07:29.952263 28453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:29.939239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33432 (local address 127.9.123.126:39333)
0926 15:07:29.939453 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:29.939459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:29.939776 (+ 317us) server_negotiation.cc:407] Connection header received
0926 15:07:29.939933 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:29.939937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:29.939999 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:29.940081 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:29.941708 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:29.942494 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:29.943854 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:29.944269 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:29.944364 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:29.947003 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:29.947008 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:29.947011 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:29.947046 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:29.949551 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:29.950292 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:29.950296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:29.950298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:29.950417 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:29.950926 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:29.950929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:29.950931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:29.951255 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:29.951454 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:29.951787 (+ 333us) server_negotiation.cc:299] Negotiation successful
0926 15:07:29.951935 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 15:07:29.956950 27993 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:07:29.964020 27993 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:07:29.988204 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:29.976796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:43021 (local address 127.0.0.1:59726)
0926 15:07:29.977874 (+ 1078us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:29.977919 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:29.978109 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:29.979497 (+ 1388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:29.979506 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:29.979551 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:29.980072 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:29.980087 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:29.982547 (+ 2460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:29.982554 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:29.985204 (+ 2650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:29.985224 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:29.987143 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:29.987150 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:29.987374 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:29.987443 (+ 69us) client_negotiation.cc:769] Sending connection context
0926 15:07:29.987601 (+ 158us) client_negotiation.cc:240] Negotiation successful
0926 15:07:29.987806 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":799,"thread_start_us":10628,"threads_started":1}
I20260926 15:07:29.989653 28467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:29.977078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59726 (local address 127.9.123.67:43021)
0926 15:07:29.978769 (+ 1691us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:29.978779 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:29.978816 (+ 37us) server_negotiation.cc:407] Connection header received
0926 15:07:29.978923 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:29.978932 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:29.979233 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:29.980314 (+ 1081us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:29.980611 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:29.982371 (+ 1760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:29.985450 (+ 3079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:29.986909 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:29.987030 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:29.987277 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:29.989078 (+ 1801us) server_negotiation.cc:299] Negotiation successful
0926 15:07:29.989296 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1506,"thread_start_us":781,"threads_started":1}
I20260926 15:07:29.993692 28386 tablet_service.cc:1467] Processing CreateTablet for tablet 255a3517c3b54736933719b73a927433 (DEFAULT_TABLE table=test-table [id=e64db4930b714251b0735ed677bf40b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:07:29.995384 28386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 255a3517c3b54736933719b73a927433. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:29.996454 28464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:29.976946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:38305 (local address 127.0.0.1:45490)
0926 15:07:29.987710 (+ 10764us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:29.987729 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:29.987857 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:29.988593 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:29.988599 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:29.988640 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:29.989024 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:29.989036 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:29.991050 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:29.991056 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:29.993010 (+ 1954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:29.993023 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:29.995725 (+ 2702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:29.995731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:29.995883 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:29.995907 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 15:07:29.995969 (+ 62us) client_negotiation.cc:240] Negotiation successful
0926 15:07:29.996116 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10617,"thread_start_us":582,"threads_started":1}
I20260926 15:07:29.996527 28466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:29.977412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45490 (local address 127.9.123.65:38305)
0926 15:07:29.986814 (+ 9402us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:29.986823 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:29.987990 (+ 1167us) server_negotiation.cc:407] Connection header received
0926 15:07:29.988094 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:29.988102 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:29.988385 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:29.989237 (+ 852us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:29.989470 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:29.990928 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:29.993195 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:29.994565 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:29.995074 (+ 509us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:29.995310 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:29.996063 (+ 753us) server_negotiation.cc:299] Negotiation successful
0926 15:07:29.996233 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9196,"thread_start_us":9003,"threads_started":1}
I20260926 15:07:29.998461 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:29.978502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:42949 (local address 127.0.0.1:41996)
0926 15:07:29.988801 (+ 10299us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:29.988818 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:29.988906 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:29.992125 (+ 3219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:29.992131 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:29.992157 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:29.992369 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:29.992379 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:29.994038 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:29.994043 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:29.995710 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:29.995721 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:29.997719 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:29.997725 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:29.997899 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:29.997933 (+ 34us) client_negotiation.cc:769] Sending connection context
0926 15:07:29.998025 (+ 92us) client_negotiation.cc:240] Negotiation successful
0926 15:07:29.998204 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10198,"thread_start_us":10744,"threads_started":1}
I20260926 15:07:29.998651 28469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:29.979014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41996 (local address 127.9.123.66:42949)
0926 15:07:29.991483 (+ 12469us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:29.991491 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:29.991518 (+ 27us) server_negotiation.cc:407] Connection header received
0926 15:07:29.991633 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:29.991642 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:29.991913 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:29.992567 (+ 654us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:29.992782 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:29.993907 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:29.995842 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:29.997550 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:29.997694 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:29.997871 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:29.998071 (+ 200us) server_negotiation.cc:299] Negotiation successful
0926 15:07:29.998241 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12268,"thread_start_us":15811,"threads_started":1}
I20260926 15:07:29.999846 28118 tablet_service.cc:1467] Processing CreateTablet for tablet 255a3517c3b54736933719b73a927433 (DEFAULT_TABLE table=test-table [id=e64db4930b714251b0735ed677bf40b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:07:30.000983 28118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 255a3517c3b54736933719b73a927433. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:30.004707 28252 tablet_service.cc:1467] Processing CreateTablet for tablet 255a3517c3b54736933719b73a927433 (DEFAULT_TABLE table=test-table [id=e64db4930b714251b0735ed677bf40b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:07:30.006135 28252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 255a3517c3b54736933719b73a927433. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:30.008147 28470 tablet_bootstrap.cc:492] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d: Bootstrap starting.
I20260926 15:07:30.011584 28470 tablet_bootstrap.cc:654] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:30.012579 28470 log.cc:824] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:30.014919 28472 tablet_bootstrap.cc:492] T 255a3517c3b54736933719b73a927433 P b4bd55c865684e2d9fc8b64a25d0d94e: Bootstrap starting.
I20260926 15:07:30.015391 28470 tablet_bootstrap.cc:492] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d: No bootstrap required, opened a new log
I20260926 15:07:30.015741 28470 ts_tablet_manager.cc:1397] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260926 15:07:30.018533 28472 tablet_bootstrap.cc:654] T 255a3517c3b54736933719b73a927433 P b4bd55c865684e2d9fc8b64a25d0d94e: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:30.019588 28473 tablet_bootstrap.cc:492] T 255a3517c3b54736933719b73a927433 P 515b40ff81f442478f09bfb70de8ee19: Bootstrap starting.
I20260926 15:07:30.019869 28472 log.cc:824] T 255a3517c3b54736933719b73a927433 P b4bd55c865684e2d9fc8b64a25d0d94e: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:30.022699 28473 tablet_bootstrap.cc:654] T 255a3517c3b54736933719b73a927433 P 515b40ff81f442478f09bfb70de8ee19: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:30.023084 28472 tablet_bootstrap.cc:492] T 255a3517c3b54736933719b73a927433 P b4bd55c865684e2d9fc8b64a25d0d94e: No bootstrap required, opened a new log
I20260926 15:07:30.023367 28472 ts_tablet_manager.cc:1397] T 255a3517c3b54736933719b73a927433 P b4bd55c865684e2d9fc8b64a25d0d94e: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260926 15:07:30.023736 28473 log.cc:824] T 255a3517c3b54736933719b73a927433 P 515b40ff81f442478f09bfb70de8ee19: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:30.026031 28470 raft_consensus.cc:348] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64c17797a8d4bc2b31e6802da89f01d" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 43021 } } peers { permanent_uuid: "b4bd55c865684e2d9fc8b64a25d0d94e" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 38305 } } peers { permanent_uuid: "515b40ff81f442478f09bfb70de8ee19" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42949 } }
I20260926 15:07:30.026433 28473 tablet_bootstrap.cc:492] T 255a3517c3b54736933719b73a927433 P 515b40ff81f442478f09bfb70de8ee19: No bootstrap required, opened a new log
I20260926 15:07:30.026679 28470 raft_consensus.cc:374] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:30.026707 28473 ts_tablet_manager.cc:1397] T 255a3517c3b54736933719b73a927433 P 515b40ff81f442478f09bfb70de8ee19: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260926 15:07:30.026840 28470 raft_consensus.cc:729] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d64c17797a8d4bc2b31e6802da89f01d, State: Initialized, Role: FOLLOWER
I20260926 15:07:30.027503 28470 consensus_queue.cc:260] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64c17797a8d4bc2b31e6802da89f01d" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 43021 } } peers { permanent_uuid: "b4bd55c865684e2d9fc8b64a25d0d94e" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 38305 } } peers { permanent_uuid: "515b40ff81f442478f09bfb70de8ee19" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42949 } }
I20260926 15:07:30.028673 28451 heartbeater.cc:503] Master 127.9.123.126:39333 was elected leader, sending a full tablet report...
I20260926 15:07:30.029325 28470 ts_tablet_manager.cc:1428] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260926 15:07:30.033635 28472 raft_consensus.cc:348] T 255a3517c3b54736933719b73a927433 P b4bd55c865684e2d9fc8b64a25d0d94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64c17797a8d4bc2b31e6802da89f01d" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 43021 } } peers { permanent_uuid: "b4bd55c865684e2d9fc8b64a25d0d94e" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 38305 } } peers { permanent_uuid: "515b40ff81f442478f09bfb70de8ee19" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42949 } }
I20260926 15:07:30.034076 28472 raft_consensus.cc:374] T 255a3517c3b54736933719b73a927433 P b4bd55c865684e2d9fc8b64a25d0d94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:30.034233 28472 raft_consensus.cc:729] T 255a3517c3b54736933719b73a927433 P b4bd55c865684e2d9fc8b64a25d0d94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4bd55c865684e2d9fc8b64a25d0d94e, State: Initialized, Role: FOLLOWER
I20260926 15:07:30.034842 28472 consensus_queue.cc:260] T 255a3517c3b54736933719b73a927433 P b4bd55c865684e2d9fc8b64a25d0d94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64c17797a8d4bc2b31e6802da89f01d" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 43021 } } peers { permanent_uuid: "b4bd55c865684e2d9fc8b64a25d0d94e" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 38305 } } peers { permanent_uuid: "515b40ff81f442478f09bfb70de8ee19" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42949 } }
I20260926 15:07:30.036087 28473 raft_consensus.cc:348] T 255a3517c3b54736933719b73a927433 P 515b40ff81f442478f09bfb70de8ee19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64c17797a8d4bc2b31e6802da89f01d" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 43021 } } peers { permanent_uuid: "b4bd55c865684e2d9fc8b64a25d0d94e" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 38305 } } peers { permanent_uuid: "515b40ff81f442478f09bfb70de8ee19" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42949 } }
I20260926 15:07:30.036567 28473 raft_consensus.cc:374] T 255a3517c3b54736933719b73a927433 P 515b40ff81f442478f09bfb70de8ee19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:30.036681 28473 raft_consensus.cc:729] T 255a3517c3b54736933719b73a927433 P 515b40ff81f442478f09bfb70de8ee19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 515b40ff81f442478f09bfb70de8ee19, State: Initialized, Role: FOLLOWER
I20260926 15:07:30.036945 28472 ts_tablet_manager.cc:1428] T 255a3517c3b54736933719b73a927433 P b4bd55c865684e2d9fc8b64a25d0d94e: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260926 15:07:30.037338 28473 consensus_queue.cc:260] T 255a3517c3b54736933719b73a927433 P 515b40ff81f442478f09bfb70de8ee19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64c17797a8d4bc2b31e6802da89f01d" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 43021 } } peers { permanent_uuid: "b4bd55c865684e2d9fc8b64a25d0d94e" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 38305 } } peers { permanent_uuid: "515b40ff81f442478f09bfb70de8ee19" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42949 } }
I20260926 15:07:30.039482 28317 heartbeater.cc:503] Master 127.9.123.126:39333 was elected leader, sending a full tablet report...
I20260926 15:07:30.040124 28473 ts_tablet_manager.cc:1428] T 255a3517c3b54736933719b73a927433 P 515b40ff81f442478f09bfb70de8ee19: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260926 15:07:30.072829 28476 raft_consensus.cc:482] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:07:30.073175 28476 raft_consensus.cc:504] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64c17797a8d4bc2b31e6802da89f01d" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 43021 } } peers { permanent_uuid: "b4bd55c865684e2d9fc8b64a25d0d94e" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 38305 } } peers { permanent_uuid: "515b40ff81f442478f09bfb70de8ee19" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42949 } }
I20260926 15:07:30.074961 28476 leader_election.cc:288] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4bd55c865684e2d9fc8b64a25d0d94e (127.9.123.65:38305), 515b40ff81f442478f09bfb70de8ee19 (127.9.123.66:42949)
I20260926 15:07:30.080505 28466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:30.075273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:35925 (local address 127.9.123.65:38305)
0926 15:07:30.075455 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:30.075460 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:30.075596 (+ 136us) server_negotiation.cc:407] Connection header received
0926 15:07:30.075778 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:30.075784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:30.075861 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:30.075960 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:30.076341 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:30.077246 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:30.078846 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:30.079620 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:30.079763 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:30.079837 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:30.080144 (+ 307us) server_negotiation.cc:299] Negotiation successful
0926 15:07:30.080266 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 15:07:30.080505 28454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:30.075272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:38305 (local address 127.9.123.67:35925)
0926 15:07:30.075478 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:30.075497 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:30.075638 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:30.076003 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:30.076008 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:30.076037 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:30.076210 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:30.076219 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:30.077391 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:30.077396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:30.078690 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:30.078703 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:30.079764 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:30.079769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:30.079990 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:30.080016 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 15:07:30.080096 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 15:07:30.080229 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":23}
I20260926 15:07:30.081781 28469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:30.076033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:43225 (local address 127.9.123.66:42949)
0926 15:07:30.076245 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:30.076251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:30.076780 (+ 529us) server_negotiation.cc:407] Connection header received
0926 15:07:30.076908 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:30.076914 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:30.076988 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:30.077074 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:30.077647 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:30.078349 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:30.079969 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:30.080862 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:30.081002 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:30.081055 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:30.081405 (+ 350us) server_negotiation.cc:299] Negotiation successful
0926 15:07:30.081537 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 15:07:30.081640 28138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "255a3517c3b54736933719b73a927433" candidate_uuid: "d64c17797a8d4bc2b31e6802da89f01d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4bd55c865684e2d9fc8b64a25d0d94e" is_pre_election: true
I20260926 15:07:30.081863 28479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:30.075710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:42949 (local address 127.9.123.67:43225)
0926 15:07:30.076674 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:30.076688 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:30.076792 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:30.077088 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:30.077091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:30.077137 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:30.077492 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:30.077501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:30.078496 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:30.078500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:30.079816 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:30.079827 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:30.081039 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:30.081044 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:30.081245 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:30.081271 (+ 26us) client_negotiation.cc:769] Sending connection context
0926 15:07:30.081361 (+ 90us) client_negotiation.cc:240] Negotiation successful
0926 15:07:30.081488 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":873,"mutex_wait_us":11,"thread_start_us":694,"threads_started":1}
I20260926 15:07:30.082264 28138 raft_consensus.cc:2454] T 255a3517c3b54736933719b73a927433 P b4bd55c865684e2d9fc8b64a25d0d94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d64c17797a8d4bc2b31e6802da89f01d in term 0.
W20260926 15:07:30.082436 28184 tablet.cc:2369] T 255a3517c3b54736933719b73a927433 P b4bd55c865684e2d9fc8b64a25d0d94e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:07:30.082875 28272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "255a3517c3b54736933719b73a927433" candidate_uuid: "d64c17797a8d4bc2b31e6802da89f01d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "515b40ff81f442478f09bfb70de8ee19" is_pre_election: true
I20260926 15:07:30.082995 28341 leader_election.cc:302] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4bd55c865684e2d9fc8b64a25d0d94e, d64c17797a8d4bc2b31e6802da89f01d; no voters:
I20260926 15:07:30.083374 28476 raft_consensus.cc:2781] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:07:30.083381 28272 raft_consensus.cc:2454] T 255a3517c3b54736933719b73a927433 P 515b40ff81f442478f09bfb70de8ee19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d64c17797a8d4bc2b31e6802da89f01d in term 0.
I20260926 15:07:30.083489 28476 raft_consensus.cc:482] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:07:30.083580 28476 raft_consensus.cc:3037] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:30.085987 28476 raft_consensus.cc:504] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64c17797a8d4bc2b31e6802da89f01d" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 43021 } } peers { permanent_uuid: "b4bd55c865684e2d9fc8b64a25d0d94e" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 38305 } } peers { permanent_uuid: "515b40ff81f442478f09bfb70de8ee19" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42949 } }
I20260926 15:07:30.086637 28476 leader_election.cc:288] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d [CANDIDATE]: Term 1 election: Requested vote from peers b4bd55c865684e2d9fc8b64a25d0d94e (127.9.123.65:38305), 515b40ff81f442478f09bfb70de8ee19 (127.9.123.66:42949)
I20260926 15:07:30.087172 28138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "255a3517c3b54736933719b73a927433" candidate_uuid: "d64c17797a8d4bc2b31e6802da89f01d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4bd55c865684e2d9fc8b64a25d0d94e"
I20260926 15:07:30.087241 28272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "255a3517c3b54736933719b73a927433" candidate_uuid: "d64c17797a8d4bc2b31e6802da89f01d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "515b40ff81f442478f09bfb70de8ee19"
I20260926 15:07:30.087446 28272 raft_consensus.cc:3037] T 255a3517c3b54736933719b73a927433 P 515b40ff81f442478f09bfb70de8ee19 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:30.087446 28138 raft_consensus.cc:3037] T 255a3517c3b54736933719b73a927433 P b4bd55c865684e2d9fc8b64a25d0d94e [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:30.090677 28138 raft_consensus.cc:2454] T 255a3517c3b54736933719b73a927433 P b4bd55c865684e2d9fc8b64a25d0d94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d64c17797a8d4bc2b31e6802da89f01d in term 1.
I20260926 15:07:30.090714 28272 raft_consensus.cc:2454] T 255a3517c3b54736933719b73a927433 P 515b40ff81f442478f09bfb70de8ee19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d64c17797a8d4bc2b31e6802da89f01d in term 1.
I20260926 15:07:30.091136 28341 leader_election.cc:302] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 515b40ff81f442478f09bfb70de8ee19, d64c17797a8d4bc2b31e6802da89f01d; no voters:
I20260926 15:07:30.091424 28476 raft_consensus.cc:2781] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:07:30.091908 28476 raft_consensus.cc:686] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d [term 1 LEADER]: Becoming Leader. State: Replica: d64c17797a8d4bc2b31e6802da89f01d, State: Running, Role: LEADER
I20260926 15:07:30.092417 28476 consensus_queue.cc:237] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64c17797a8d4bc2b31e6802da89f01d" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 43021 } } peers { permanent_uuid: "b4bd55c865684e2d9fc8b64a25d0d94e" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 38305 } } peers { permanent_uuid: "515b40ff81f442478f09bfb70de8ee19" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42949 } }
I20260926 15:07:30.097740 27994 catalog_manager.cc:5495] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d reported cstate change: term changed from 0 to 1, leader changed from <none> to d64c17797a8d4bc2b31e6802da89f01d (127.9.123.67). New cstate: current_term: 1 leader_uuid: "d64c17797a8d4bc2b31e6802da89f01d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64c17797a8d4bc2b31e6802da89f01d" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 43021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4bd55c865684e2d9fc8b64a25d0d94e" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 38305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "515b40ff81f442478f09bfb70de8ee19" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42949 } health_report { overall_health: UNKNOWN } } }
W20260926 15:07:30.120306 28318 tablet.cc:2369] T 255a3517c3b54736933719b73a927433 P 515b40ff81f442478f09bfb70de8ee19: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:07:30.124137 28467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:30.118824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59742 (local address 127.9.123.67:43021)
0926 15:07:30.118996 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:30.119000 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:30.119016 (+ 16us) server_negotiation.cc:407] Connection header received
0926 15:07:30.119099 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:30.119110 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:30.119196 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:30.119316 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:07:30.119704 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:30.120462 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:30.122066 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:30.122459 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:30.122552 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:30.123013 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:07:30.123516 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:07:30.123642 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:30.123833 (+ 191us) server_negotiation.cc:299] Negotiation successful
0926 15:07:30.123878 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260926 15:07:30.132611 28272 raft_consensus.cc:1261] T 255a3517c3b54736933719b73a927433 P 515b40ff81f442478f09bfb70de8ee19 [term 1 FOLLOWER]: Refusing update from remote peer d64c17797a8d4bc2b31e6802da89f01d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:07:30.132611 28138 raft_consensus.cc:1261] T 255a3517c3b54736933719b73a927433 P b4bd55c865684e2d9fc8b64a25d0d94e [term 1 FOLLOWER]: Refusing update from remote peer d64c17797a8d4bc2b31e6802da89f01d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:07:30.133767 28476 consensus_queue.cc:1035] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d [LEADER]: Connected to new peer: Peer: permanent_uuid: "515b40ff81f442478f09bfb70de8ee19" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 42949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:07:30.134125 28480 consensus_queue.cc:1035] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4bd55c865684e2d9fc8b64a25d0d94e" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 38305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:07:30.147187 28485 mvcc.cc:205] Tried to move back new op lower bound from 7333622784532959232 to 7333622784382697472. Current Snapshot: MvccSnapshot[applied={T|T < 7333622784532959232}]
I20260926 15:07:30.161280 27994 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:07:30.161566 27994 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:07:30.165561 27994 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a093ff918cb347ab92bd43f77afb3b7b: Sending DeleteTablet for 3 replicas of tablet 255a3517c3b54736933719b73a927433
I20260926 15:07:30.166890 28386 tablet_service.cc:1514] Processing DeleteTablet for tablet 255a3517c3b54736933719b73a927433 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:07:30 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:59726
I20260926 15:07:30.166980 28118 tablet_service.cc:1514] Processing DeleteTablet for tablet 255a3517c3b54736933719b73a927433 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:07:30 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:45490
I20260926 15:07:30.167028 28252 tablet_service.cc:1514] Processing DeleteTablet for tablet 255a3517c3b54736933719b73a927433 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:07:30 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:41996
I20260926 15:07:30.171895 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 28055
I20260926 15:07:30.182473 28495 tablet_replica.cc:330] T 255a3517c3b54736933719b73a927433 P 515b40ff81f442478f09bfb70de8ee19: stopping tablet replica
I20260926 15:07:30.183350 28495 raft_consensus.cc:2229] T 255a3517c3b54736933719b73a927433 P 515b40ff81f442478f09bfb70de8ee19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:07:30.184068 28495 raft_consensus.cc:2258] T 255a3517c3b54736933719b73a927433 P 515b40ff81f442478f09bfb70de8ee19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:07:30.186522 28495 ts_tablet_manager.cc:1905] T 255a3517c3b54736933719b73a927433 P 515b40ff81f442478f09bfb70de8ee19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:07:30.194159 28495 ts_tablet_manager.cc:1918] T 255a3517c3b54736933719b73a927433 P 515b40ff81f442478f09bfb70de8ee19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:07:30.194468 28495 log.cc:1196] T 255a3517c3b54736933719b73a927433 P 515b40ff81f442478f09bfb70de8ee19: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/255a3517c3b54736933719b73a927433
I20260926 15:07:30.195163 28495 ts_tablet_manager.cc:1939] T 255a3517c3b54736933719b73a927433 P 515b40ff81f442478f09bfb70de8ee19: Deleting consensus metadata
I20260926 15:07:30.197364 27980 catalog_manager.cc:4841] TS 515b40ff81f442478f09bfb70de8ee19 (127.9.123.66:42949): tablet 255a3517c3b54736933719b73a927433 (table test-table [id=e64db4930b714251b0735ed677bf40b0]) successfully deleted
I20260926 15:07:30.199879 28493 tablet_replica.cc:330] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d: stopping tablet replica
I20260926 15:07:30.200484 28493 raft_consensus.cc:2229] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:07:30.201650 28493 raft_consensus.cc:2258] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:07:30.202485 28494 tablet_replica.cc:330] T 255a3517c3b54736933719b73a927433 P b4bd55c865684e2d9fc8b64a25d0d94e: stopping tablet replica
I20260926 15:07:30.202979 28494 raft_consensus.cc:2229] T 255a3517c3b54736933719b73a927433 P b4bd55c865684e2d9fc8b64a25d0d94e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:07:30.203552 28494 raft_consensus.cc:2258] T 255a3517c3b54736933719b73a927433 P b4bd55c865684e2d9fc8b64a25d0d94e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:07:30.203876 28493 ts_tablet_manager.cc:1905] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:07:30.205829 28494 ts_tablet_manager.cc:1905] T 255a3517c3b54736933719b73a927433 P b4bd55c865684e2d9fc8b64a25d0d94e: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 26 15:07:30 dist-test-slave-7153 krb5kdc[27949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.65@KRBTEST.COM
I20260926 15:07:30.212616 28493 ts_tablet_manager.cc:1918] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:07:30.212925 28493 log.cc:1196] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/255a3517c3b54736933719b73a927433
I20260926 15:07:30.213212 28494 ts_tablet_manager.cc:1918] T 255a3517c3b54736933719b73a927433 P b4bd55c865684e2d9fc8b64a25d0d94e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:07:30.213420 28494 log.cc:1196] T 255a3517c3b54736933719b73a927433 P b4bd55c865684e2d9fc8b64a25d0d94e: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790434939095847-9709-0/minicluster-data/ts-0/wal/wals/255a3517c3b54736933719b73a927433
I20260926 15:07:30.213719 28493 ts_tablet_manager.cc:1939] T 255a3517c3b54736933719b73a927433 P d64c17797a8d4bc2b31e6802da89f01d: Deleting consensus metadata
I20260926 15:07:30.213994 28494 ts_tablet_manager.cc:1939] T 255a3517c3b54736933719b73a927433 P b4bd55c865684e2d9fc8b64a25d0d94e: Deleting consensus metadata
I20260926 15:07:30.215716 27979 catalog_manager.cc:4841] TS d64c17797a8d4bc2b31e6802da89f01d (127.9.123.67:43021): tablet 255a3517c3b54736933719b73a927433 (table test-table [id=e64db4930b714251b0735ed677bf40b0]) successfully deleted
I20260926 15:07:30.216171 27980 catalog_manager.cc:4841] TS b4bd55c865684e2d9fc8b64a25d0d94e (127.9.123.65:38305): tablet 255a3517c3b54736933719b73a927433 (table test-table [id=e64db4930b714251b0735ed677bf40b0]) successfully deleted
I20260926 15:07:30.218879 28466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:30.173648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45496 (local address 127.9.123.65:38305)
0926 15:07:30.173857 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:30.173864 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:30.198993 (+ 25129us) server_negotiation.cc:407] Connection header received
0926 15:07:30.199251 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:30.199258 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:30.199358 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:30.199450 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:30.201478 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:30.202707 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:30.204566 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:30.205246 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:30.205365 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:30.211103 (+ 5738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:30.211122 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:30.211142 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:30.211197 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:30.215278 (+ 4081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:30.216499 (+ 1221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:30.216510 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:30.216513 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:30.216696 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:30.217218 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:30.217223 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:30.217227 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:30.217674 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:30.217973 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:30.218294 (+ 321us) server_negotiation.cc:299] Negotiation successful
0926 15:07:30.218522 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 15:07:30.219472 28178 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:30.314471 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 28055
I20260926 15:07:30.329376 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 28189
Sep 26 15:07:30 dist-test-slave-7153 krb5kdc[27949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.66@KRBTEST.COM
I20260926 15:07:30.347450 28469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:30.330222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42006 (local address 127.9.123.66:42949)
0926 15:07:30.330410 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:30.330416 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:30.330437 (+ 21us) server_negotiation.cc:407] Connection header received
0926 15:07:30.330532 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:30.330538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:30.330618 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:30.330730 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:30.332074 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:30.333240 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:30.334702 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:30.335361 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:30.335500 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:30.340347 (+ 4847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:30.340368 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:30.340387 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:30.340441 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:30.344196 (+ 3755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:30.345049 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:30.345060 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:30.345063 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:30.345259 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:30.345672 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:30.345698 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:30.345702 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:30.346212 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:30.346496 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:30.346845 (+ 349us) server_negotiation.cc:299] Negotiation successful
0926 15:07:30.347076 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260926 15:07:30.348259 28312 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:30.486778 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 28189
I20260926 15:07:30.500869 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 28323
Sep 26 15:07:30 dist-test-slave-7153 krb5kdc[27949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.67@KRBTEST.COM
I20260926 15:07:30.520247 28467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:30.501668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59756 (local address 127.9.123.67:43021)
0926 15:07:30.501815 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:30.501820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:30.501841 (+ 21us) server_negotiation.cc:407] Connection header received
0926 15:07:30.502060 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:30.502064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:30.502126 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:30.502212 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:30.503927 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:30.504759 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:30.506771 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:30.507431 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:30.507591 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:30.513897 (+ 6306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:30.513912 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:30.513928 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:30.513970 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:30.517028 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:30.517836 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:30.517846 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:30.517850 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:30.518091 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:30.518464 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:30.518469 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:30.518472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:30.518966 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:30.519291 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:30.519582 (+ 291us) server_negotiation.cc:299] Negotiation successful
0926 15:07:30.519818 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260926 15:07:30.521126 28446 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:30.643294 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 28323
I20260926 15:07:30.659430 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 27962
I20260926 15:07:30.660494 28024 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:30.770411 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 27962
2026-09-26T15:07:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4251 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-26T15:07:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:07:30Z Disabled control of system clock
I20260926 15:07:30.841413 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:40899
--webserver_interface=127.9.123.126
--webserver_port=0
--builtin_ntp_servers=127.9.123.84:34947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:40899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 15:07:31.172780 28504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:31.173137 28504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:31.173245 28504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:31.180788 28504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:07:31.180917 28504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:31.180955 28504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:31.181012 28504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:07:31.181056 28504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:07:31.190791 28504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:34947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:40899
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:40899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:31.192093 28504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:31.194031 28504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:31.206069 28509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:31.206125 28510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:31.206231 28512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:31.206959 28504 server_base.cc:1034] running on GCE node
I20260926 15:07:31.207831 28504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:31.209412 28504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:31.210629 28504 hybrid_clock.cc:648] HybridClock initialized: now 1790435251210560 us; error 55 us; skew 500 ppm
I20260926 15:07:31.211004 28504 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:07:31.213578 28504 webserver.cc:460] Webserver started at http://127.9.123.126:38521/ using document root <none> and password file <none>
I20260926 15:07:31.214310 28504 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:31.214402 28504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:31.214722 28504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:31.217792 28504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "8841a0629d9a47d48973e4b30e0e68d8"
format_stamp: "Formatted at 2026-09-26 15:07:31 on dist-test-slave-7153"
server_key: "be888a7c37cb058e4609c4c2ebc2e461"
server_key_iv: "3b89d1daf15d955780c0768b0f90f3fd"
server_key_version: "encryptionkey@0"
I20260926 15:07:31.218655 28504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "8841a0629d9a47d48973e4b30e0e68d8"
format_stamp: "Formatted at 2026-09-26 15:07:31 on dist-test-slave-7153"
server_key: "be888a7c37cb058e4609c4c2ebc2e461"
server_key_iv: "3b89d1daf15d955780c0768b0f90f3fd"
server_key_version: "encryptionkey@0"
I20260926 15:07:31.224291 28504 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 15:07:31.228171 28518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:31.229801 28504 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 15:07:31.230013 28504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "8841a0629d9a47d48973e4b30e0e68d8"
format_stamp: "Formatted at 2026-09-26 15:07:31 on dist-test-slave-7153"
server_key: "be888a7c37cb058e4609c4c2ebc2e461"
server_key_iv: "3b89d1daf15d955780c0768b0f90f3fd"
server_key_version: "encryptionkey@0"
I20260926 15:07:31.230257 28504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:31.258339 28504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:31.259299 28504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:31.259599 28504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:31.277647 28504 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:40899
I20260926 15:07:31.277673 28569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:40899 every 8 connection(s)
I20260926 15:07:31.279028 28504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:07:31.282902 28570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:31.285082 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 28504
I20260926 15:07:31.285305 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:07:31.285820 9709 external_mini_cluster.cc:1428] Setting key 94a2a0561de12fa46c23eee8c1e8ce4b
I20260926 15:07:31.295145 28570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8841a0629d9a47d48973e4b30e0e68d8: Bootstrap starting.
I20260926 15:07:31.298696 28570 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8841a0629d9a47d48973e4b30e0e68d8: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:31.299705 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:31.287816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48600 (local address 127.9.123.126:40899)
0926 15:07:31.288903 (+ 1087us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:31.288918 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:31.288962 (+ 44us) server_negotiation.cc:407] Connection header received
0926 15:07:31.290715 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:31.290740 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:31.291225 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:31.291580 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:31.292170 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:31.293550 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:31.295366 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:31.296047 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:31.296203 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:31.297428 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:31.297436 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:31.297450 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:31.297977 (+ 527us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:31.298067 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:31.298087 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:31.298184 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:31.298382 (+ 198us) server_negotiation.cc:299] Negotiation successful
0926 15:07:31.298707 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":381,"threads_started":1}
I20260926 15:07:31.299782 28570 log.cc:824] T 00000000000000000000000000000000 P 8841a0629d9a47d48973e4b30e0e68d8: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:31.302990 28570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8841a0629d9a47d48973e4b30e0e68d8: No bootstrap required, opened a new log
I20260926 15:07:31.310307 28570 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8841a0629d9a47d48973e4b30e0e68d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8841a0629d9a47d48973e4b30e0e68d8" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 40899 } }
I20260926 15:07:31.310698 28570 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8841a0629d9a47d48973e4b30e0e68d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:31.310794 28570 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8841a0629d9a47d48973e4b30e0e68d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8841a0629d9a47d48973e4b30e0e68d8, State: Initialized, Role: FOLLOWER
I20260926 15:07:31.311336 28570 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8841a0629d9a47d48973e4b30e0e68d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8841a0629d9a47d48973e4b30e0e68d8" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 40899 } }
I20260926 15:07:31.311544 28570 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8841a0629d9a47d48973e4b30e0e68d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:07:31.311677 28570 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8841a0629d9a47d48973e4b30e0e68d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:07:31.311831 28570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8841a0629d9a47d48973e4b30e0e68d8 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:31.314024 28570 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8841a0629d9a47d48973e4b30e0e68d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8841a0629d9a47d48973e4b30e0e68d8" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 40899 } }
I20260926 15:07:31.314554 28570 leader_election.cc:302] T 00000000000000000000000000000000 P 8841a0629d9a47d48973e4b30e0e68d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8841a0629d9a47d48973e4b30e0e68d8; no voters:
I20260926 15:07:31.315055 28570 leader_election.cc:288] T 00000000000000000000000000000000 P 8841a0629d9a47d48973e4b30e0e68d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:07:31.315202 28575 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8841a0629d9a47d48973e4b30e0e68d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:07:31.315852 28575 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8841a0629d9a47d48973e4b30e0e68d8 [term 1 LEADER]: Becoming Leader. State: Replica: 8841a0629d9a47d48973e4b30e0e68d8, State: Running, Role: LEADER
I20260926 15:07:31.316365 28575 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8841a0629d9a47d48973e4b30e0e68d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8841a0629d9a47d48973e4b30e0e68d8" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 40899 } }
I20260926 15:07:31.317003 28570 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8841a0629d9a47d48973e4b30e0e68d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:07:31.318912 28577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8841a0629d9a47d48973e4b30e0e68d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8841a0629d9a47d48973e4b30e0e68d8. Latest consensus state: current_term: 1 leader_uuid: "8841a0629d9a47d48973e4b30e0e68d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8841a0629d9a47d48973e4b30e0e68d8" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 40899 } } }
I20260926 15:07:31.319329 28577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8841a0629d9a47d48973e4b30e0e68d8 [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:31.319571 28576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8841a0629d9a47d48973e4b30e0e68d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8841a0629d9a47d48973e4b30e0e68d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8841a0629d9a47d48973e4b30e0e68d8" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 40899 } } }
I20260926 15:07:31.319938 28576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8841a0629d9a47d48973e4b30e0e68d8 [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:31.321986 28581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:07:31.327365 28581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:07:31.336812 28581 catalog_manager.cc:1347] Generated new cluster ID: 614ba527048a4ee3b16da9ca8bb0f6e6
I20260926 15:07:31.336934 28581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:07:31.353729 28581 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:07:31.355645 28581 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:07:31.370762 28581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8841a0629d9a47d48973e4b30e0e68d8: Generated new TSK 0
I20260926 15:07:31.371888 28581 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:07:31.384654 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:40899
--builtin_ntp_servers=127.9.123.84:34947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 15:07:31.725704 28594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:31.726006 28594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:31.726118 28594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:31.733948 28594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:31.734057 28594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:31.734193 28594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:07:31.743703 28594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:34947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:40899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:31.744984 28594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:31.746855 28594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:31.760097 28602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:31.760143 28599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:31.760646 28600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:31.760887 28594 server_base.cc:1034] running on GCE node
I20260926 15:07:31.761507 28594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:31.762475 28594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:31.763705 28594 hybrid_clock.cc:648] HybridClock initialized: now 1790435251763658 us; error 49 us; skew 500 ppm
I20260926 15:07:31.764055 28594 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:07:31.766487 28594 webserver.cc:460] Webserver started at http://127.9.123.65:38071/ using document root <none> and password file <none>
I20260926 15:07:31.767199 28594 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:31.767294 28594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:31.767611 28594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:31.770658 28594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "b522e19020e04b7a84f8150152c403ae"
format_stamp: "Formatted at 2026-09-26 15:07:31 on dist-test-slave-7153"
server_key: "b8d71e94317af261ecf789160a841b2a"
server_key_iv: "9b8bac954053d3e1dc92dd26384a401e"
server_key_version: "encryptionkey@0"
I20260926 15:07:31.771446 28594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "b522e19020e04b7a84f8150152c403ae"
format_stamp: "Formatted at 2026-09-26 15:07:31 on dist-test-slave-7153"
server_key: "b8d71e94317af261ecf789160a841b2a"
server_key_iv: "9b8bac954053d3e1dc92dd26384a401e"
server_key_version: "encryptionkey@0"
I20260926 15:07:31.776872 28594 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 15:07:31.780561 28608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:31.781980 28594 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:07:31.782158 28594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "b522e19020e04b7a84f8150152c403ae"
format_stamp: "Formatted at 2026-09-26 15:07:31 on dist-test-slave-7153"
server_key: "b8d71e94317af261ecf789160a841b2a"
server_key_iv: "9b8bac954053d3e1dc92dd26384a401e"
server_key_version: "encryptionkey@0"
I20260926 15:07:31.782372 28594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:31.793865 28594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:31.794694 28594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:31.794979 28594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:31.796010 28594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:07:31.797609 28594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:07:31.797701 28594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:31.797785 28594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:07:31.797838 28594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:31.826330 28594 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:45143
I20260926 15:07:31.826382 28720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:45143 every 8 connection(s)
I20260926 15:07:31.827600 28594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:07:31.832860 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 28594
I20260926 15:07:31.833026 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:07:31.833459 9709 external_mini_cluster.cc:1428] Setting key 92fd34be1b50d84bc6dda33c20ae3100
I20260926 15:07:31.836074 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:40899
--builtin_ntp_servers=127.9.123.84:34947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 15:07:31.840297 28723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:31.830178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:35675 (local address 127.9.123.126:40899)
0926 15:07:31.830610 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:31.830616 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:31.831824 (+ 1208us) server_negotiation.cc:407] Connection header received
0926 15:07:31.833698 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:31.833705 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:31.833820 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:31.833910 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:31.835168 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:31.835959 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:31.837607 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:31.837993 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:31.838103 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:31.839423 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:31.839428 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:31.839431 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:31.839449 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:31.839502 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:31.839517 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:31.839608 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:31.839952 (+ 344us) server_negotiation.cc:299] Negotiation successful
0926 15:07:31.840043 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":199,"threads_started":1}
I20260926 15:07:31.841765 28724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:31.830793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:40899 (local address 127.9.123.65:35675)
0926 15:07:31.831674 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:31.831714 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:31.833381 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:31.834173 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:31.834181 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:31.834281 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:07:31.834999 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:31.835021 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:31.836105 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:31.836109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:31.837460 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:31.837473 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:31.838212 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:31.838219 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:31.838442 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:31.839093 (+ 651us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 15:07:31.839095 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:07:31.839141 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 15:07:31.839235 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:07:31.839650 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:07:31.839664 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:07:31.839673 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 15:07:31.839914 (+ 241us) client_negotiation.cc:240] Negotiation successful
0926 15:07:31.840394 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":268,"threads_started":1}
I20260926 15:07:31.843417 28721 heartbeater.cc:347] Connected to a master server at 127.9.123.126:40899
I20260926 15:07:31.843732 28721 heartbeater.cc:464] Registering TS with master...
I20260926 15:07:31.844539 28721 heartbeater.cc:511] Master 127.9.123.126:40899 requested a full tablet report, sending...
I20260926 15:07:31.846872 28535 ts_manager.cc:194] Registered new tserver with Master: b522e19020e04b7a84f8150152c403ae (127.9.123.65:45143)
I20260926 15:07:31.848361 28535 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.123.65:35675
W20260926 15:07:32.165740 28725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:32.166060 28725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:32.166181 28725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:32.173659 28725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:32.173772 28725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:32.173911 28725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:07:32.183159 28725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:34947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:40899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:32.184495 28725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:32.186367 28725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:32.199343 28731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:32.199267 28730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:32.199375 28733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:32.200173 28725 server_base.cc:1034] running on GCE node
I20260926 15:07:32.200812 28725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:32.201851 28725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:32.203075 28725 hybrid_clock.cc:648] HybridClock initialized: now 1790435252202996 us; error 57 us; skew 500 ppm
I20260926 15:07:32.203387 28725 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:07:32.205693 28725 webserver.cc:460] Webserver started at http://127.9.123.66:43797/ using document root <none> and password file <none>
I20260926 15:07:32.206384 28725 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:32.206471 28725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:32.206768 28725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:32.209679 28725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "5eaa379e54d04b4391a4a4a36bb83e20"
format_stamp: "Formatted at 2026-09-26 15:07:32 on dist-test-slave-7153"
server_key: "1107e44c536330e4ade5bb20c4801837"
server_key_iv: "cef27f3400804636cd8ed6204e783e5e"
server_key_version: "encryptionkey@0"
I20260926 15:07:32.210454 28725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "5eaa379e54d04b4391a4a4a36bb83e20"
format_stamp: "Formatted at 2026-09-26 15:07:32 on dist-test-slave-7153"
server_key: "1107e44c536330e4ade5bb20c4801837"
server_key_iv: "cef27f3400804636cd8ed6204e783e5e"
server_key_version: "encryptionkey@0"
I20260926 15:07:32.215516 28725 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260926 15:07:32.218992 28739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:32.220319 28725 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 15:07:32.220479 28725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "5eaa379e54d04b4391a4a4a36bb83e20"
format_stamp: "Formatted at 2026-09-26 15:07:32 on dist-test-slave-7153"
server_key: "1107e44c536330e4ade5bb20c4801837"
server_key_iv: "cef27f3400804636cd8ed6204e783e5e"
server_key_version: "encryptionkey@0"
I20260926 15:07:32.220693 28725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:32.239399 28725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:32.240325 28725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:32.240643 28725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:32.241770 28725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:07:32.243263 28725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:07:32.243363 28725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:32.243474 28725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:07:32.243552 28725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:32.274165 28725 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:33365
I20260926 15:07:32.274215 28851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:33365 every 8 connection(s)
I20260926 15:07:32.275449 28725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:07:32.281893 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 28725
I20260926 15:07:32.282092 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:07:32.282603 9709 external_mini_cluster.cc:1428] Setting key 3b2dce6679491ace87cf910aeeaa321d
I20260926 15:07:32.285597 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:40899
--builtin_ntp_servers=127.9.123.84:34947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 15:07:32.290216 28723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:32.278076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:35671 (local address 127.9.123.126:40899)
0926 15:07:32.278242 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:32.278247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:32.279908 (+ 1661us) server_negotiation.cc:407] Connection header received
0926 15:07:32.281846 (+ 1938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:32.281853 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:32.281934 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:32.282019 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:32.283333 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:32.284497 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:32.286994 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:32.287625 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:32.287757 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:32.289349 (+ 1592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:32.289356 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:32.289360 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:32.289381 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:32.289422 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:32.289438 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:32.289543 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:32.289789 (+ 246us) server_negotiation.cc:299] Negotiation successful
0926 15:07:32.289901 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260926 15:07:32.291594 28854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:32.278744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:40899 (local address 127.9.123.66:35671)
0926 15:07:32.279727 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:32.279781 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:32.281564 (+ 1783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:32.282387 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:32.282396 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:32.282493 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:07:32.283128 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:32.283153 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:32.284687 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:32.284692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:32.286838 (+ 2146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:32.286856 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:32.287877 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:32.287885 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:32.288162 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:32.288956 (+ 794us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 15:07:32.288959 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:07:32.289013 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 15:07:32.289145 (+ 132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:07:32.289546 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:07:32.289555 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:07:32.289563 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 15:07:32.289737 (+ 174us) client_negotiation.cc:240] Negotiation successful
0926 15:07:32.290161 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":303,"threads_started":1}
I20260926 15:07:32.293324 28852 heartbeater.cc:347] Connected to a master server at 127.9.123.126:40899
I20260926 15:07:32.293671 28852 heartbeater.cc:464] Registering TS with master...
I20260926 15:07:32.294552 28852 heartbeater.cc:511] Master 127.9.123.126:40899 requested a full tablet report, sending...
I20260926 15:07:32.295953 28535 ts_manager.cc:194] Registered new tserver with Master: 5eaa379e54d04b4391a4a4a36bb83e20 (127.9.123.66:33365)
I20260926 15:07:32.296748 28535 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.123.66:35671
W20260926 15:07:32.612021 28855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:32.612309 28855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:32.612419 28855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:32.619819 28855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:32.619930 28855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:32.620074 28855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:07:32.629387 28855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:34947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:40899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:32.630707 28855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:32.632505 28855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:32.645691 28861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:32.645792 28860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:32.645913 28863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:32.646797 28855 server_base.cc:1034] running on GCE node
I20260926 15:07:32.647439 28855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:32.648433 28855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:32.649667 28855 hybrid_clock.cc:648] HybridClock initialized: now 1790435252649612 us; error 59 us; skew 500 ppm
I20260926 15:07:32.649982 28855 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:07:32.652220 28855 webserver.cc:460] Webserver started at http://127.9.123.67:45145/ using document root <none> and password file <none>
I20260926 15:07:32.652911 28855 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:32.653008 28855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:32.653360 28855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:32.656203 28855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "01a7bef70a2e4d498c039cc327033da3"
format_stamp: "Formatted at 2026-09-26 15:07:32 on dist-test-slave-7153"
server_key: "7c141d306c075350db049b9fa32191ab"
server_key_iv: "c192da33733ae775d654b11cbaf9166f"
server_key_version: "encryptionkey@0"
I20260926 15:07:32.656975 28855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "01a7bef70a2e4d498c039cc327033da3"
format_stamp: "Formatted at 2026-09-26 15:07:32 on dist-test-slave-7153"
server_key: "7c141d306c075350db049b9fa32191ab"
server_key_iv: "c192da33733ae775d654b11cbaf9166f"
server_key_version: "encryptionkey@0"
I20260926 15:07:32.662029 28855 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260926 15:07:32.665606 28869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:32.667007 28855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260926 15:07:32.667166 28855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "01a7bef70a2e4d498c039cc327033da3"
format_stamp: "Formatted at 2026-09-26 15:07:32 on dist-test-slave-7153"
server_key: "7c141d306c075350db049b9fa32191ab"
server_key_iv: "c192da33733ae775d654b11cbaf9166f"
server_key_version: "encryptionkey@0"
I20260926 15:07:32.667382 28855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:32.689065 28855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:32.689953 28855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:32.690237 28855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:32.691217 28855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:07:32.692739 28855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:07:32.692834 28855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:32.692942 28855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:07:32.693008 28855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:32.723042 28855 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:33945
I20260926 15:07:32.723124 28981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:33945 every 8 connection(s)
I20260926 15:07:32.724339 28855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:07:32.732517 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 28855
I20260926 15:07:32.732692 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:07:32.733137 9709 external_mini_cluster.cc:1428] Setting key 563e371a462d797af12eb1b5890bbb81
I20260926 15:07:32.737914 28723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:32.727040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:51685 (local address 127.9.123.126:40899)
0926 15:07:32.727235 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:32.727239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:32.728697 (+ 1458us) server_negotiation.cc:407] Connection header received
0926 15:07:32.730618 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:32.730624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:32.730709 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:32.730799 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:32.731977 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:32.732785 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:32.734701 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:32.735252 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:32.735411 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:32.737011 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:32.737017 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:32.737021 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:32.737043 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:32.737078 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:32.737091 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:32.737233 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:32.737440 (+ 207us) server_negotiation.cc:299] Negotiation successful
0926 15:07:32.737540 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 15:07:32.738951 28984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:32.727570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:40899 (local address 127.9.123.67:51685)
0926 15:07:32.728520 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:32.728561 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:32.730297 (+ 1736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:32.731067 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:32.731076 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:32.731185 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:07:32.731796 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:32.731816 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:32.732930 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:32.732934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:32.734548 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:32.734563 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:32.735516 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:32.735523 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:32.735787 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:32.736667 (+ 880us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 15:07:32.736670 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:07:32.736716 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 15:07:32.736813 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:07:32.737242 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:07:32.737252 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:07:32.737258 (+ 6us) client_negotiation.cc:769] Sending connection context
0926 15:07:32.737427 (+ 169us) client_negotiation.cc:240] Negotiation successful
0926 15:07:32.737798 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":276,"threads_started":1}
I20260926 15:07:32.740638 28982 heartbeater.cc:347] Connected to a master server at 127.9.123.126:40899
I20260926 15:07:32.740931 28982 heartbeater.cc:464] Registering TS with master...
I20260926 15:07:32.741775 28982 heartbeater.cc:511] Master 127.9.123.126:40899 requested a full tablet report, sending...
I20260926 15:07:32.743254 28535 ts_manager.cc:194] Registered new tserver with Master: 01a7bef70a2e4d498c039cc327033da3 (127.9.123.67:33945)
I20260926 15:07:32.744127 28535 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.123.67:51685
I20260926 15:07:32.748920 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 15:07:32.759786 28723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:32.753333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48604 (local address 127.9.123.126:40899)
0926 15:07:32.753514 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:32.753521 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:32.753903 (+ 382us) server_negotiation.cc:407] Connection header received
0926 15:07:32.754075 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:32.754081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:32.754166 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:32.754256 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:32.754774 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:32.755492 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:32.757085 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:32.757677 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:32.757796 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:32.759015 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:32.759020 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:32.759023 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:32.759044 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:32.759079 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:32.759093 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:32.759252 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:32.759440 (+ 188us) server_negotiation.cc:299] Negotiation successful
0926 15:07:32.759502 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 15:07:32.764459 28535 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48604:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:07:32.772007 28535 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:07:32.797206 28997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:32.786374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57142 (local address 127.9.123.66:33365)
0926 15:07:32.789426 (+ 3052us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:32.789436 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:32.789486 (+ 50us) server_negotiation.cc:407] Connection header received
0926 15:07:32.789581 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:32.789588 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:32.789792 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:32.790156 (+ 364us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:32.790588 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:32.792058 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:32.794492 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:32.795991 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:32.796121 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:32.796399 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:32.796660 (+ 261us) server_negotiation.cc:299] Negotiation successful
0926 15:07:32.796834 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2875,"thread_start_us":2752,"threads_started":1}
I20260926 15:07:32.797268 28994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:32.785795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:33365 (local address 127.0.0.1:57142)
0926 15:07:32.787396 (+ 1601us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:32.787414 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:32.787536 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:32.789980 (+ 2444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:32.789984 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:32.790003 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:32.790418 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:32.790430 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:32.792235 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:32.792242 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:32.794290 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:32.794302 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:32.796185 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:32.796192 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:32.796391 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:32.796444 (+ 53us) client_negotiation.cc:769] Sending connection context
0926 15:07:32.796595 (+ 151us) client_negotiation.cc:240] Negotiation successful
0926 15:07:32.796821 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1467,"mutex_wait_us":43,"thread_start_us":399,"threads_started":1}
I20260926 15:07:32.797824 28995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:32.785464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:45143 (local address 127.0.0.1:57020)
0926 15:07:32.786616 (+ 1152us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:32.786654 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:32.786813 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:32.789439 (+ 2626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:32.789467 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:32.789498 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:32.789951 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:32.789967 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:32.792121 (+ 2154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:32.792127 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:32.794179 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:32.794194 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:32.796792 (+ 2598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:32.796798 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:32.796966 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:32.797001 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 15:07:32.797066 (+ 65us) client_negotiation.cc:240] Negotiation successful
0926 15:07:32.797244 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":882,"mutex_wait_us":52,"thread_start_us":1617,"threads_started":1}
I20260926 15:07:32.798301 28998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:32.787042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:33945 (local address 127.0.0.1:55714)
0926 15:07:32.788019 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:32.788035 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:32.788144 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:32.789936 (+ 1792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:32.789941 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:32.789966 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:32.790907 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:32.790919 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:32.793744 (+ 2825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:32.793750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:32.795791 (+ 2041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:32.795806 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:32.797606 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:32.797613 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:32.797795 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:32.797830 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 15:07:32.797902 (+ 72us) client_negotiation.cc:240] Negotiation successful
0926 15:07:32.798029 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"thread_start_us":850,"threads_started":1}
I20260926 15:07:32.799057 28996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:32.786067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57020 (local address 127.9.123.65:45143)
0926 15:07:32.788682 (+ 2615us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:32.788693 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:32.788730 (+ 37us) server_negotiation.cc:407] Connection header received
0926 15:07:32.788847 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:32.788855 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:32.789163 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:32.789422 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:32.790203 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:32.791878 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:32.794436 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:32.796576 (+ 2140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:32.798360 (+ 1784us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:32.798594 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:32.798678 (+ 84us) server_negotiation.cc:299] Negotiation successful
0926 15:07:32.798808 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2441,"thread_start_us":2284,"threads_started":1}
I20260926 15:07:32.799242 28999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:32.787613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55714 (local address 127.9.123.67:33945)
0926 15:07:32.789181 (+ 1568us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:32.789190 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:32.789225 (+ 35us) server_negotiation.cc:407] Connection header received
0926 15:07:32.789333 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:32.789341 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:32.789613 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:32.789927 (+ 314us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:32.791463 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:32.793582 (+ 2119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:32.796021 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:32.797414 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:32.797870 (+ 456us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:32.798161 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:32.798716 (+ 555us) server_negotiation.cc:299] Negotiation successful
0926 15:07:32.798893 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1351,"thread_start_us":1186,"threads_started":1}
I20260926 15:07:32.801769 28787 tablet_service.cc:1467] Processing CreateTablet for tablet 9de7a8c11ed4494b94068d48106a8151 (DEFAULT_TABLE table=test-table [id=df35a18cb06543188615d91cf0432db4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:07:32.802381 28656 tablet_service.cc:1467] Processing CreateTablet for tablet 9de7a8c11ed4494b94068d48106a8151 (DEFAULT_TABLE table=test-table [id=df35a18cb06543188615d91cf0432db4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:07:32.803326 28787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9de7a8c11ed4494b94068d48106a8151. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:32.802886 28917 tablet_service.cc:1467] Processing CreateTablet for tablet 9de7a8c11ed4494b94068d48106a8151 (DEFAULT_TABLE table=test-table [id=df35a18cb06543188615d91cf0432db4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:07:32.803454 28656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9de7a8c11ed4494b94068d48106a8151. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:32.804205 28917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9de7a8c11ed4494b94068d48106a8151. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:32.815194 29000 tablet_bootstrap.cc:492] T 9de7a8c11ed4494b94068d48106a8151 P b522e19020e04b7a84f8150152c403ae: Bootstrap starting.
I20260926 15:07:32.819028 29002 tablet_bootstrap.cc:492] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3: Bootstrap starting.
I20260926 15:07:32.819123 29000 tablet_bootstrap.cc:654] T 9de7a8c11ed4494b94068d48106a8151 P b522e19020e04b7a84f8150152c403ae: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:32.819325 29001 tablet_bootstrap.cc:492] T 9de7a8c11ed4494b94068d48106a8151 P 5eaa379e54d04b4391a4a4a36bb83e20: Bootstrap starting.
I20260926 15:07:32.820477 29000 log.cc:824] T 9de7a8c11ed4494b94068d48106a8151 P b522e19020e04b7a84f8150152c403ae: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:32.822588 29001 tablet_bootstrap.cc:654] T 9de7a8c11ed4494b94068d48106a8151 P 5eaa379e54d04b4391a4a4a36bb83e20: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:32.822597 29002 tablet_bootstrap.cc:654] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:32.823637 29001 log.cc:824] T 9de7a8c11ed4494b94068d48106a8151 P 5eaa379e54d04b4391a4a4a36bb83e20: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:32.823798 29002 log.cc:824] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:32.823976 29000 tablet_bootstrap.cc:492] T 9de7a8c11ed4494b94068d48106a8151 P b522e19020e04b7a84f8150152c403ae: No bootstrap required, opened a new log
I20260926 15:07:32.824297 29000 ts_tablet_manager.cc:1397] T 9de7a8c11ed4494b94068d48106a8151 P b522e19020e04b7a84f8150152c403ae: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260926 15:07:32.827431 29002 tablet_bootstrap.cc:492] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3: No bootstrap required, opened a new log
I20260926 15:07:32.827850 29002 ts_tablet_manager.cc:1397] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260926 15:07:32.835569 29001 tablet_bootstrap.cc:492] T 9de7a8c11ed4494b94068d48106a8151 P 5eaa379e54d04b4391a4a4a36bb83e20: No bootstrap required, opened a new log
I20260926 15:07:32.835563 29000 raft_consensus.cc:348] T 9de7a8c11ed4494b94068d48106a8151 P b522e19020e04b7a84f8150152c403ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b522e19020e04b7a84f8150152c403ae" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45143 } } peers { permanent_uuid: "5eaa379e54d04b4391a4a4a36bb83e20" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33365 } } peers { permanent_uuid: "01a7bef70a2e4d498c039cc327033da3" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33945 } }
I20260926 15:07:32.835938 29001 ts_tablet_manager.cc:1397] T 9de7a8c11ed4494b94068d48106a8151 P 5eaa379e54d04b4391a4a4a36bb83e20: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.005s
I20260926 15:07:32.836117 29000 raft_consensus.cc:374] T 9de7a8c11ed4494b94068d48106a8151 P b522e19020e04b7a84f8150152c403ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:32.836261 29000 raft_consensus.cc:729] T 9de7a8c11ed4494b94068d48106a8151 P b522e19020e04b7a84f8150152c403ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b522e19020e04b7a84f8150152c403ae, State: Initialized, Role: FOLLOWER
I20260926 15:07:32.836975 29000 consensus_queue.cc:260] T 9de7a8c11ed4494b94068d48106a8151 P b522e19020e04b7a84f8150152c403ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b522e19020e04b7a84f8150152c403ae" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45143 } } peers { permanent_uuid: "5eaa379e54d04b4391a4a4a36bb83e20" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33365 } } peers { permanent_uuid: "01a7bef70a2e4d498c039cc327033da3" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33945 } }
I20260926 15:07:32.838990 29002 raft_consensus.cc:348] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b522e19020e04b7a84f8150152c403ae" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45143 } } peers { permanent_uuid: "5eaa379e54d04b4391a4a4a36bb83e20" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33365 } } peers { permanent_uuid: "01a7bef70a2e4d498c039cc327033da3" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33945 } }
I20260926 15:07:32.839501 29002 raft_consensus.cc:374] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:32.839630 29002 raft_consensus.cc:729] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a7bef70a2e4d498c039cc327033da3, State: Initialized, Role: FOLLOWER
I20260926 15:07:32.840431 29002 consensus_queue.cc:260] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b522e19020e04b7a84f8150152c403ae" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45143 } } peers { permanent_uuid: "5eaa379e54d04b4391a4a4a36bb83e20" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33365 } } peers { permanent_uuid: "01a7bef70a2e4d498c039cc327033da3" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33945 } }
I20260926 15:07:32.845006 28982 heartbeater.cc:503] Master 127.9.123.126:40899 was elected leader, sending a full tablet report...
I20260926 15:07:32.845228 29000 ts_tablet_manager.cc:1428] T 9de7a8c11ed4494b94068d48106a8151 P b522e19020e04b7a84f8150152c403ae: Time spent starting tablet: real 0.021s user 0.019s sys 0.000s
I20260926 15:07:32.845713 29002 ts_tablet_manager.cc:1428] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3: Time spent starting tablet: real 0.018s user 0.017s sys 0.000s
I20260926 15:07:32.845747 28721 heartbeater.cc:503] Master 127.9.123.126:40899 was elected leader, sending a full tablet report...
I20260926 15:07:32.848711 29001 raft_consensus.cc:348] T 9de7a8c11ed4494b94068d48106a8151 P 5eaa379e54d04b4391a4a4a36bb83e20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b522e19020e04b7a84f8150152c403ae" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45143 } } peers { permanent_uuid: "5eaa379e54d04b4391a4a4a36bb83e20" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33365 } } peers { permanent_uuid: "01a7bef70a2e4d498c039cc327033da3" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33945 } }
I20260926 15:07:32.849159 29001 raft_consensus.cc:374] T 9de7a8c11ed4494b94068d48106a8151 P 5eaa379e54d04b4391a4a4a36bb83e20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:32.849308 29001 raft_consensus.cc:729] T 9de7a8c11ed4494b94068d48106a8151 P 5eaa379e54d04b4391a4a4a36bb83e20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eaa379e54d04b4391a4a4a36bb83e20, State: Initialized, Role: FOLLOWER
I20260926 15:07:32.849987 29001 consensus_queue.cc:260] T 9de7a8c11ed4494b94068d48106a8151 P 5eaa379e54d04b4391a4a4a36bb83e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b522e19020e04b7a84f8150152c403ae" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45143 } } peers { permanent_uuid: "5eaa379e54d04b4391a4a4a36bb83e20" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33365 } } peers { permanent_uuid: "01a7bef70a2e4d498c039cc327033da3" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33945 } }
I20260926 15:07:32.851212 28852 heartbeater.cc:503] Master 127.9.123.126:40899 was elected leader, sending a full tablet report...
I20260926 15:07:32.851948 29001 ts_tablet_manager.cc:1428] T 9de7a8c11ed4494b94068d48106a8151 P 5eaa379e54d04b4391a4a4a36bb83e20: Time spent starting tablet: real 0.016s user 0.012s sys 0.000s
W20260926 15:07:32.976079 28983 tablet.cc:2369] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:07:33.004108 29007 raft_consensus.cc:482] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:07:33.004369 29007 raft_consensus.cc:504] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b522e19020e04b7a84f8150152c403ae" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45143 } } peers { permanent_uuid: "5eaa379e54d04b4391a4a4a36bb83e20" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33365 } } peers { permanent_uuid: "01a7bef70a2e4d498c039cc327033da3" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33945 } }
I20260926 15:07:33.006107 29007 leader_election.cc:288] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b522e19020e04b7a84f8150152c403ae (127.9.123.65:45143), 5eaa379e54d04b4391a4a4a36bb83e20 (127.9.123.66:33365)
I20260926 15:07:33.011448 28984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:33.006403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:45143 (local address 127.9.123.67:41067)
0926 15:07:33.006588 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:33.006605 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:33.006735 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:33.007171 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:33.007176 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:33.007209 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:33.007363 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:33.007374 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:33.008333 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:33.008338 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:33.009705 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:33.009718 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:33.010744 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:33.010749 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:33.010922 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:33.010945 (+ 23us) client_negotiation.cc:769] Sending connection context
0926 15:07:33.011020 (+ 75us) client_negotiation.cc:240] Negotiation successful
0926 15:07:33.011166 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260926 15:07:33.011439 28996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:33.006609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:41067 (local address 127.9.123.65:45143)
0926 15:07:33.006797 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:33.006803 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:33.006822 (+ 19us) server_negotiation.cc:407] Connection header received
0926 15:07:33.006892 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:33.006896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:33.006974 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:33.007066 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:33.007528 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:33.008209 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:33.009862 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:33.010606 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:33.010751 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:33.010822 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:33.011064 (+ 242us) server_negotiation.cc:299] Negotiation successful
0926 15:07:33.011188 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 15:07:33.012149 28997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:33.006722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:34057 (local address 127.9.123.66:33365)
0926 15:07:33.006965 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:33.006971 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:33.007387 (+ 416us) server_negotiation.cc:407] Connection header received
0926 15:07:33.007562 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:33.007567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:33.007631 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:33.007710 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:33.008189 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:33.008875 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:33.010576 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:33.011319 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:33.011454 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:33.011530 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:33.011830 (+ 300us) server_negotiation.cc:299] Negotiation successful
0926 15:07:33.011958 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260926 15:07:33.012176 29009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:33.006435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:33365 (local address 127.9.123.67:34057)
0926 15:07:33.007284 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:33.007298 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:33.007412 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:33.007739 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:33.007743 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:33.007766 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:33.008052 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:33.008060 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:33.009022 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:33.009025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:33.010403 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:33.010416 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:33.011482 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:33.011488 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:33.011662 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:33.011697 (+ 35us) client_negotiation.cc:769] Sending connection context
0926 15:07:33.011776 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 15:07:33.011890 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":459,"threads_started":1}
I20260926 15:07:33.012547 28676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9de7a8c11ed4494b94068d48106a8151" candidate_uuid: "01a7bef70a2e4d498c039cc327033da3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b522e19020e04b7a84f8150152c403ae" is_pre_election: true
I20260926 15:07:33.013051 28676 raft_consensus.cc:2454] T 9de7a8c11ed4494b94068d48106a8151 P b522e19020e04b7a84f8150152c403ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01a7bef70a2e4d498c039cc327033da3 in term 0.
I20260926 15:07:33.013183 28807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9de7a8c11ed4494b94068d48106a8151" candidate_uuid: "01a7bef70a2e4d498c039cc327033da3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5eaa379e54d04b4391a4a4a36bb83e20" is_pre_election: true
I20260926 15:07:33.013651 28807 raft_consensus.cc:2454] T 9de7a8c11ed4494b94068d48106a8151 P 5eaa379e54d04b4391a4a4a36bb83e20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01a7bef70a2e4d498c039cc327033da3 in term 0.
I20260926 15:07:33.013693 28870 leader_election.cc:302] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a7bef70a2e4d498c039cc327033da3, b522e19020e04b7a84f8150152c403ae; no voters:
I20260926 15:07:33.014087 29007 raft_consensus.cc:2781] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:07:33.014235 29007 raft_consensus.cc:482] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:07:33.014339 29007 raft_consensus.cc:3037] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:33.016391 29007 raft_consensus.cc:504] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b522e19020e04b7a84f8150152c403ae" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45143 } } peers { permanent_uuid: "5eaa379e54d04b4391a4a4a36bb83e20" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33365 } } peers { permanent_uuid: "01a7bef70a2e4d498c039cc327033da3" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33945 } }
I20260926 15:07:33.016976 29007 leader_election.cc:288] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3 [CANDIDATE]: Term 1 election: Requested vote from peers b522e19020e04b7a84f8150152c403ae (127.9.123.65:45143), 5eaa379e54d04b4391a4a4a36bb83e20 (127.9.123.66:33365)
I20260926 15:07:33.017508 28807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9de7a8c11ed4494b94068d48106a8151" candidate_uuid: "01a7bef70a2e4d498c039cc327033da3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5eaa379e54d04b4391a4a4a36bb83e20"
I20260926 15:07:33.017556 28676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9de7a8c11ed4494b94068d48106a8151" candidate_uuid: "01a7bef70a2e4d498c039cc327033da3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b522e19020e04b7a84f8150152c403ae"
I20260926 15:07:33.017692 28807 raft_consensus.cc:3037] T 9de7a8c11ed4494b94068d48106a8151 P 5eaa379e54d04b4391a4a4a36bb83e20 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:33.017758 28676 raft_consensus.cc:3037] T 9de7a8c11ed4494b94068d48106a8151 P b522e19020e04b7a84f8150152c403ae [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:33.019979 28676 raft_consensus.cc:2454] T 9de7a8c11ed4494b94068d48106a8151 P b522e19020e04b7a84f8150152c403ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a7bef70a2e4d498c039cc327033da3 in term 1.
I20260926 15:07:33.019990 28807 raft_consensus.cc:2454] T 9de7a8c11ed4494b94068d48106a8151 P 5eaa379e54d04b4391a4a4a36bb83e20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a7bef70a2e4d498c039cc327033da3 in term 1.
I20260926 15:07:33.020397 28870 leader_election.cc:302] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a7bef70a2e4d498c039cc327033da3, b522e19020e04b7a84f8150152c403ae; no voters:
I20260926 15:07:33.020694 29007 raft_consensus.cc:2781] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:07:33.021154 29007 raft_consensus.cc:686] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3 [term 1 LEADER]: Becoming Leader. State: Replica: 01a7bef70a2e4d498c039cc327033da3, State: Running, Role: LEADER
I20260926 15:07:33.021677 29007 consensus_queue.cc:237] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b522e19020e04b7a84f8150152c403ae" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45143 } } peers { permanent_uuid: "5eaa379e54d04b4391a4a4a36bb83e20" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33365 } } peers { permanent_uuid: "01a7bef70a2e4d498c039cc327033da3" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33945 } }
I20260926 15:07:33.026221 28535 catalog_manager.cc:5495] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01a7bef70a2e4d498c039cc327033da3 (127.9.123.67). New cstate: current_term: 1 leader_uuid: "01a7bef70a2e4d498c039cc327033da3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b522e19020e04b7a84f8150152c403ae" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5eaa379e54d04b4391a4a4a36bb83e20" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01a7bef70a2e4d498c039cc327033da3" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 33945 } health_report { overall_health: HEALTHY } } }
W20260926 15:07:33.027388 28853 tablet.cc:2369] T 9de7a8c11ed4494b94068d48106a8151 P 5eaa379e54d04b4391a4a4a36bb83e20: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 15:07:33.081051 28722 tablet.cc:2369] T 9de7a8c11ed4494b94068d48106a8151 P b522e19020e04b7a84f8150152c403ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:07:33.099435 28999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:33.093493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55730 (local address 127.9.123.67:33945)
0926 15:07:33.093696 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:33.093702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:33.093721 (+ 19us) server_negotiation.cc:407] Connection header received
0926 15:07:33.093895 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:33.093901 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:33.093993 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:33.094134 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:07:33.094542 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:33.095325 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:33.096867 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:33.097386 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:33.097539 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:33.098140 (+ 601us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:07:33.098710 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:07:33.098849 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:33.099112 (+ 263us) server_negotiation.cc:299] Negotiation successful
0926 15:07:33.099166 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260926 15:07:33.108721 28807 raft_consensus.cc:1261] T 9de7a8c11ed4494b94068d48106a8151 P 5eaa379e54d04b4391a4a4a36bb83e20 [term 1 FOLLOWER]: Refusing update from remote peer 01a7bef70a2e4d498c039cc327033da3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:07:33.108721 28676 raft_consensus.cc:1261] T 9de7a8c11ed4494b94068d48106a8151 P b522e19020e04b7a84f8150152c403ae [term 1 FOLLOWER]: Refusing update from remote peer 01a7bef70a2e4d498c039cc327033da3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:07:33.109922 29010 consensus_queue.cc:1035] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b522e19020e04b7a84f8150152c403ae" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 45143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:07:33.110332 29007 consensus_queue.cc:1035] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5eaa379e54d04b4391a4a4a36bb83e20" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 33365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:07:33.122043 29013 mvcc.cc:205] Tried to move back new op lower bound from 7333622796721602560 to 7333622796380860416. Current Snapshot: MvccSnapshot[applied={T|T < 7333622796721602560}]
I20260926 15:07:33.122359 29014 mvcc.cc:205] Tried to move back new op lower bound from 7333622796721602560 to 7333622796380860416. Current Snapshot: MvccSnapshot[applied={T|T < 7333622796721602560}]
I20260926 15:07:33.139297 28535 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:07:33.139540 28535 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:07:33.143131 28535 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8841a0629d9a47d48973e4b30e0e68d8: Sending DeleteTablet for 3 replicas of tablet 9de7a8c11ed4494b94068d48106a8151
I20260926 15:07:33.144272 28656 tablet_service.cc:1514] Processing DeleteTablet for tablet 9de7a8c11ed4494b94068d48106a8151 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:07:33 UTC) from {username='slave'} at 127.0.0.1:57020
I20260926 15:07:33.144487 28787 tablet_service.cc:1514] Processing DeleteTablet for tablet 9de7a8c11ed4494b94068d48106a8151 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:07:33 UTC) from {username='slave'} at 127.0.0.1:57142
I20260926 15:07:33.144709 28917 tablet_service.cc:1514] Processing DeleteTablet for tablet 9de7a8c11ed4494b94068d48106a8151 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:07:33 UTC) from {username='slave'} at 127.0.0.1:55714
I20260926 15:07:33.150506 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 28594
I20260926 15:07:33.157972 29023 tablet_replica.cc:330] T 9de7a8c11ed4494b94068d48106a8151 P b522e19020e04b7a84f8150152c403ae: stopping tablet replica
I20260926 15:07:33.158581 29024 tablet_replica.cc:330] T 9de7a8c11ed4494b94068d48106a8151 P 5eaa379e54d04b4391a4a4a36bb83e20: stopping tablet replica
I20260926 15:07:33.158996 29024 raft_consensus.cc:2229] T 9de7a8c11ed4494b94068d48106a8151 P 5eaa379e54d04b4391a4a4a36bb83e20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:07:33.159628 29023 raft_consensus.cc:2229] T 9de7a8c11ed4494b94068d48106a8151 P b522e19020e04b7a84f8150152c403ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:07:33.160238 29024 raft_consensus.cc:2258] T 9de7a8c11ed4494b94068d48106a8151 P 5eaa379e54d04b4391a4a4a36bb83e20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:07:33.161036 29023 raft_consensus.cc:2258] T 9de7a8c11ed4494b94068d48106a8151 P b522e19020e04b7a84f8150152c403ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:07:33.163411 28996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:33.151814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57022 (local address 127.9.123.65:45143)
0926 15:07:33.152016 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:33.152023 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:33.152937 (+ 914us) server_negotiation.cc:407] Connection header received
0926 15:07:33.153180 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:33.153186 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:33.153322 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:33.153447 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:33.155412 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:33.156611 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:33.159052 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:33.159717 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:33.159849 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:33.161250 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:33.161353 (+ 103us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:33.161372 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:33.162148 (+ 776us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:33.162218 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:33.162245 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:33.162331 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:33.162967 (+ 636us) server_negotiation.cc:299] Negotiation successful
0926 15:07:33.163088 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 15:07:33.164058 29024 ts_tablet_manager.cc:1905] T 9de7a8c11ed4494b94068d48106a8151 P 5eaa379e54d04b4391a4a4a36bb83e20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:07:33.164748 28716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:33.169087 29025 tablet_replica.cc:330] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3: stopping tablet replica
I20260926 15:07:33.169710 29025 raft_consensus.cc:2229] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3 [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:07:33.170868 29025 raft_consensus.cc:2258] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:07:33.173259 29025 ts_tablet_manager.cc:1905] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:07:33.173928 29024 ts_tablet_manager.cc:1918] T 9de7a8c11ed4494b94068d48106a8151 P 5eaa379e54d04b4391a4a4a36bb83e20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:07:33.174616 29024 log.cc:1196] T 9de7a8c11ed4494b94068d48106a8151 P 5eaa379e54d04b4391a4a4a36bb83e20: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/9de7a8c11ed4494b94068d48106a8151
I20260926 15:07:33.175280 29024 ts_tablet_manager.cc:1939] T 9de7a8c11ed4494b94068d48106a8151 P 5eaa379e54d04b4391a4a4a36bb83e20: Deleting consensus metadata
I20260926 15:07:33.177315 28521 catalog_manager.cc:4841] TS 5eaa379e54d04b4391a4a4a36bb83e20 (127.9.123.66:33365): tablet 9de7a8c11ed4494b94068d48106a8151 (table test-table [id=df35a18cb06543188615d91cf0432db4]) successfully deleted
I20260926 15:07:33.181365 29025 ts_tablet_manager.cc:1918] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:07:33.181574 29025 log.cc:1196] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/9de7a8c11ed4494b94068d48106a8151
I20260926 15:07:33.182122 29025 ts_tablet_manager.cc:1939] T 9de7a8c11ed4494b94068d48106a8151 P 01a7bef70a2e4d498c039cc327033da3: Deleting consensus metadata
I20260926 15:07:33.184064 28521 catalog_manager.cc:4841] TS 01a7bef70a2e4d498c039cc327033da3 (127.9.123.67:33945): tablet 9de7a8c11ed4494b94068d48106a8151 (table test-table [id=df35a18cb06543188615d91cf0432db4]) successfully deleted
I20260926 15:07:33.263559 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 28594
W20260926 15:07:33.280928 28519 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.123.65:45143 (error 108)
I20260926 15:07:33.287597 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 28725
I20260926 15:07:33.296958 28997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:33.288766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57156 (local address 127.9.123.66:33365)
0926 15:07:33.288951 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:33.288957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:33.288978 (+ 21us) server_negotiation.cc:407] Connection header received
0926 15:07:33.289039 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:33.289044 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:33.289197 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:33.289327 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:33.289803 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:33.290968 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:33.292788 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:33.293446 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:33.293561 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:33.294956 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:33.294967 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:33.294997 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:33.295751 (+ 754us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:33.295814 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:33.295837 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:33.295933 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:33.296501 (+ 568us) server_negotiation.cc:299] Negotiation successful
0926 15:07:33.296615 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260926 15:07:33.297668 28847 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:33.301321 28998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:33.300823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:45143 (local address 127.0.0.1:57026)
0926 15:07:33.301002 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:33.301162 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:45143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260926 15:07:33.301595 28519 catalog_manager.cc:4584] TS b522e19020e04b7a84f8150152c403ae (127.9.123.65:45143): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9de7a8c11ed4494b94068d48106a8151: Network error: Client connection negotiation failed: client connection to 127.9.123.65:45143: connect: Connection refused (error 111)
I20260926 15:07:33.355288 28998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:33.354826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:45143 (local address 127.0.0.1:57040)
0926 15:07:33.355046 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:33.355137 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:45143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260926 15:07:33.356364 28998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:33.356010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:45143 (local address 127.0.0.1:57042)
0926 15:07:33.356161 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:33.356232 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:45143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260926 15:07:33.416844 28998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:33.416338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:45143 (local address 127.0.0.1:57058)
0926 15:07:33.416541 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:33.416637 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:45143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260926 15:07:33.418028 28998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:33.417659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:45143 (local address 127.0.0.1:57068)
0926 15:07:33.417817 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:33.417896 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:45143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260926 15:07:33.450461 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 28725
I20260926 15:07:33.471482 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 28855
I20260926 15:07:33.479363 28999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:33.472292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55736 (local address 127.9.123.67:33945)
0926 15:07:33.472484 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:33.472489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:33.472530 (+ 41us) server_negotiation.cc:407] Connection header received
0926 15:07:33.472746 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:33.472751 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:33.472857 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:33.472941 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:33.473423 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:33.474189 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:33.476090 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:33.476545 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:33.476643 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:33.477859 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:33.477868 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:33.477883 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:33.478526 (+ 643us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:33.478582 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:33.478610 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:33.478705 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:33.478899 (+ 194us) server_negotiation.cc:299] Negotiation successful
0926 15:07:33.479008 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 15:07:33.480064 28977 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:33.498761 28998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:33.498254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:45143 (local address 127.0.0.1:57072)
0926 15:07:33.498473 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:33.498577 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:45143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260926 15:07:33.499799 28998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:33.499424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:45143 (local address 127.0.0.1:57076)
0926 15:07:33.499608 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:33.499682 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:45143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260926 15:07:33.602821 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 28855
I20260926 15:07:33.618733 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 28504
I20260926 15:07:33.619807 28565 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:33.726845 28998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:33.726284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:45143 (local address 127.0.0.1:57086)
0926 15:07:33.726503 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:33.726640 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.65:45143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260926 15:07:33.726919 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 28504
2026-09-26T15:07:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2956 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:07:33 dist-test-slave-7153 krb5kdc[29034](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:07:33 dist-test-slave-7153 krb5kdc[29034](info): set up 2 sockets
Sep 26 15:07:33 dist-test-slave-7153 krb5kdc[29034](info): commencing operation
krb5kdc: starting...
W20260926 15:07:34.745545 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.981s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:07:34 dist-test-slave-7153 krb5kdc[29034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:07:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:07:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:07:34.993420 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:37549
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:40369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:37549
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:07:35.326874 29047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:35.327248 29047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:35.327395 29047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:35.335340 29047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:07:35.335505 29047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:35.335582 29047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:35.335660 29047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:07:35.335726 29047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:07:35.346899 29047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:40369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:37549
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:37549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:35.348382 29047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:35.349843 29047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:35.362112 29053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:35.362192 29055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:35.362237 29052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:35.363060 29047 server_base.cc:1034] running on GCE node
I20260926 15:07:35.363970 29047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:35.365582 29047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:35.366827 29047 hybrid_clock.cc:648] HybridClock initialized: now 1790435255366765 us; error 45 us; skew 500 ppm
I20260926 15:07:35.367205 29047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:35 dist-test-slave-7153 krb5kdc[29034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435255, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:35.377895 29047 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:07:35.379703 29047 webserver.cc:460] Webserver started at http://127.9.123.126:44803/ using document root <none> and password file <none>
I20260926 15:07:35.380450 29047 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:35.380589 29047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:35.380896 29047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:35.384021 29047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "363e3edb0ded4aac91d4610f466e0bbb"
format_stamp: "Formatted at 2026-09-26 15:07:35 on dist-test-slave-7153"
server_key: "28900129d7731629cc9b58de8936aa8c"
server_key_iv: "fb60b5062653fdf16a8cc01bcf9b9440"
server_key_version: "encryptionkey@0"
I20260926 15:07:35.384811 29047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "363e3edb0ded4aac91d4610f466e0bbb"
format_stamp: "Formatted at 2026-09-26 15:07:35 on dist-test-slave-7153"
server_key: "28900129d7731629cc9b58de8936aa8c"
server_key_iv: "fb60b5062653fdf16a8cc01bcf9b9440"
server_key_version: "encryptionkey@0"
I20260926 15:07:35.390424 29047 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 15:07:35.394213 29062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:35.395648 29047 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 15:07:35.395828 29047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "363e3edb0ded4aac91d4610f466e0bbb"
format_stamp: "Formatted at 2026-09-26 15:07:35 on dist-test-slave-7153"
server_key: "28900129d7731629cc9b58de8936aa8c"
server_key_iv: "fb60b5062653fdf16a8cc01bcf9b9440"
server_key_version: "encryptionkey@0"
I20260926 15:07:35.396054 29047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:35.415138 29047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:35.419754 29047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:35.420046 29047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:35.438488 29047 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:37549
I20260926 15:07:35.438510 29113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:37549 every 8 connection(s)
I20260926 15:07:35.440166 29047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:07:35.444049 29114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:35.448163 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 29047
I20260926 15:07:35.448343 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:07:35.448899 9709 external_mini_cluster.cc:1428] Setting key 02ba2b03fd593c03e6b172f4a31c80a6
I20260926 15:07:35.457935 29114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 363e3edb0ded4aac91d4610f466e0bbb: Bootstrap starting.
I20260926 15:07:35.461906 29114 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 363e3edb0ded4aac91d4610f466e0bbb: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:35.463275 29114 log.cc:824] T 00000000000000000000000000000000 P 363e3edb0ded4aac91d4610f466e0bbb: Log is configured to *not* fsync() on all Append() calls
Sep 26 15:07:35 dist-test-slave-7153 krb5kdc[29034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:07:35.466902 29114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 363e3edb0ded4aac91d4610f466e0bbb: No bootstrap required, opened a new log
I20260926 15:07:35.473794 29117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:35.451041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41426 (local address 127.9.123.126:37549)
0926 15:07:35.451856 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:35.451870 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:35.451915 (+ 45us) server_negotiation.cc:407] Connection header received
0926 15:07:35.453668 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:35.453697 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:35.454239 (+ 542us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:35.454630 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:35.456394 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:35.457951 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:35.459785 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:35.460579 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:35.460736 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:35.466005 (+ 5269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:35.466013 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:35.466029 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:35.466073 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:35.469123 (+ 3050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:35.470126 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:35.470133 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:35.470140 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:35.470378 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:35.470815 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:35.470819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:35.470822 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:35.471452 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:35.471702 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:35.472191 (+ 489us) server_negotiation.cc:299] Negotiation successful
0926 15:07:35.472650 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":358,"threads_started":1}
I20260926 15:07:35.476064 29114 raft_consensus.cc:348] T 00000000000000000000000000000000 P 363e3edb0ded4aac91d4610f466e0bbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "363e3edb0ded4aac91d4610f466e0bbb" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 37549 } }
I20260926 15:07:35.476459 29114 raft_consensus.cc:374] T 00000000000000000000000000000000 P 363e3edb0ded4aac91d4610f466e0bbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:35.476559 29114 raft_consensus.cc:729] T 00000000000000000000000000000000 P 363e3edb0ded4aac91d4610f466e0bbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 363e3edb0ded4aac91d4610f466e0bbb, State: Initialized, Role: FOLLOWER
I20260926 15:07:35.477250 29114 consensus_queue.cc:260] T 00000000000000000000000000000000 P 363e3edb0ded4aac91d4610f466e0bbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "363e3edb0ded4aac91d4610f466e0bbb" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 37549 } }
I20260926 15:07:35.477496 29114 raft_consensus.cc:388] T 00000000000000000000000000000000 P 363e3edb0ded4aac91d4610f466e0bbb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:07:35.477623 29114 raft_consensus.cc:482] T 00000000000000000000000000000000 P 363e3edb0ded4aac91d4610f466e0bbb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:07:35.477769 29114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 363e3edb0ded4aac91d4610f466e0bbb [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:35.479957 29114 raft_consensus.cc:504] T 00000000000000000000000000000000 P 363e3edb0ded4aac91d4610f466e0bbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "363e3edb0ded4aac91d4610f466e0bbb" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 37549 } }
I20260926 15:07:35.480480 29114 leader_election.cc:302] T 00000000000000000000000000000000 P 363e3edb0ded4aac91d4610f466e0bbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 363e3edb0ded4aac91d4610f466e0bbb; no voters:
I20260926 15:07:35.481161 29114 leader_election.cc:288] T 00000000000000000000000000000000 P 363e3edb0ded4aac91d4610f466e0bbb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:07:35.481303 29119 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 363e3edb0ded4aac91d4610f466e0bbb [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:07:35.482331 29119 raft_consensus.cc:686] T 00000000000000000000000000000000 P 363e3edb0ded4aac91d4610f466e0bbb [term 1 LEADER]: Becoming Leader. State: Replica: 363e3edb0ded4aac91d4610f466e0bbb, State: Running, Role: LEADER
I20260926 15:07:35.482848 29119 consensus_queue.cc:237] T 00000000000000000000000000000000 P 363e3edb0ded4aac91d4610f466e0bbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "363e3edb0ded4aac91d4610f466e0bbb" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 37549 } }
I20260926 15:07:35.483331 29114 sys_catalog.cc:564] T 00000000000000000000000000000000 P 363e3edb0ded4aac91d4610f466e0bbb [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:07:35.485527 29121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 363e3edb0ded4aac91d4610f466e0bbb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 363e3edb0ded4aac91d4610f466e0bbb. Latest consensus state: current_term: 1 leader_uuid: "363e3edb0ded4aac91d4610f466e0bbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "363e3edb0ded4aac91d4610f466e0bbb" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 37549 } } }
I20260926 15:07:35.486006 29121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 363e3edb0ded4aac91d4610f466e0bbb [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:35.485630 29120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 363e3edb0ded4aac91d4610f466e0bbb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "363e3edb0ded4aac91d4610f466e0bbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "363e3edb0ded4aac91d4610f466e0bbb" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 37549 } } }
I20260926 15:07:35.486577 29120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 363e3edb0ded4aac91d4610f466e0bbb [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:35.486968 29128 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:07:35.492859 29128 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:07:35.502522 29128 catalog_manager.cc:1347] Generated new cluster ID: 81ac0ef644004e82aa1b5bd4ce0123bb
I20260926 15:07:35.502655 29128 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:07:35.517944 29128 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:07:35.519235 29128 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:07:35.531812 29128 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 363e3edb0ded4aac91d4610f466e0bbb: Generated new TSK 0
I20260926 15:07:35.532970 29128 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.123.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
Entry for principal kudu/127.9.123.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab.
I20260926 15:07:35.602304 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:37549
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:40369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:07:35.940467 29140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:35.940783 29140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:35.940901 29140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:35.948314 29140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:35.948448 29140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:35.948611 29140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:07:35.958801 29140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:40369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.65.keytab
--principal=kudu/127.9.123.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:37549
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:35.960161 29140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:35.961520 29140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:35.974784 29145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:35.974814 29146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:35.975072 29148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:35.976539 29140 server_base.cc:1034] running on GCE node
I20260926 15:07:35.977246 29140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:35.978243 29140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:35.979398 29140 hybrid_clock.cc:648] HybridClock initialized: now 1790435255979331 us; error 61 us; skew 500 ppm
I20260926 15:07:35.979727 29140 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:35 dist-test-slave-7153 krb5kdc[29034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435255, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:35.982933 29140 init.cc:377] Logged in from keytab as kudu/127.9.123.65@KRBTEST.COM (short username kudu)
I20260926 15:07:35.984516 29140 webserver.cc:460] Webserver started at http://127.9.123.65:46107/ using document root <none> and password file <none>
I20260926 15:07:35.985265 29140 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:35.985386 29140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:35.985677 29140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:35.988654 29140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "8b5051b8df88410ba4c9ab19ceb63353"
format_stamp: "Formatted at 2026-09-26 15:07:35 on dist-test-slave-7153"
server_key: "380d998ff13fa1365e13e7ea0c418f63"
server_key_iv: "4129183ec54d09eedf8516e71d3f4ddb"
server_key_version: "encryptionkey@0"
I20260926 15:07:35.989495 29140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "8b5051b8df88410ba4c9ab19ceb63353"
format_stamp: "Formatted at 2026-09-26 15:07:35 on dist-test-slave-7153"
server_key: "380d998ff13fa1365e13e7ea0c418f63"
server_key_iv: "4129183ec54d09eedf8516e71d3f4ddb"
server_key_version: "encryptionkey@0"
I20260926 15:07:35.994654 29140 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260926 15:07:35.998255 29155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:35.999687 29140 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 15:07:35.999843 29140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "8b5051b8df88410ba4c9ab19ceb63353"
format_stamp: "Formatted at 2026-09-26 15:07:35 on dist-test-slave-7153"
server_key: "380d998ff13fa1365e13e7ea0c418f63"
server_key_iv: "4129183ec54d09eedf8516e71d3f4ddb"
server_key_version: "encryptionkey@0"
I20260926 15:07:36.000070 29140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:36.015427 29140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:36.019804 29140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:36.020058 29140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:36.021219 29140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:07:36.022539 29140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:07:36.022615 29140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:36.022706 29140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:07:36.022746 29140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:36.053576 29140 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:35603
I20260926 15:07:36.053617 29267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:35603 every 8 connection(s)
I20260926 15:07:36.054849 29140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:07:36.059747 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 29140
I20260926 15:07:36.059938 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:07:36.060494 9709 external_mini_cluster.cc:1428] Setting key 1227b3a5db158b1c7439cdc0266ba549
Sep 26 15:07:36 dist-test-slave-7153 krb5kdc[29034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435255, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.65@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:07:36.077189 29270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:36.057402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:45649 (local address 127.9.123.126:37549)
0926 15:07:36.057799 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:36.057804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:36.058878 (+ 1074us) server_negotiation.cc:407] Connection header received
0926 15:07:36.061319 (+ 2441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:36.061325 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:36.061442 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:36.061536 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:36.063354 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:36.064135 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:36.066262 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:36.066674 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:36.066771 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:36.070997 (+ 4226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:36.071002 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:36.071006 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:36.071041 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:36.073917 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:36.074749 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:36.074755 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:36.074758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:36.074909 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:36.075307 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:36.075312 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:36.075315 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:36.075651 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:36.075784 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:36.076581 (+ 797us) server_negotiation.cc:299] Negotiation successful
0926 15:07:36.076809 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":188,"threads_started":1}
I20260926 15:07:36.078109 29271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:36.057991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:37549 (local address 127.9.123.65:45649)
0926 15:07:36.058707 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:36.058748 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:36.060960 (+ 2212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:36.061805 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:36.061813 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:36.062348 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:07:36.063169 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:36.063197 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:36.064287 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:36.064293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:36.066106 (+ 1813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:36.066122 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:36.066902 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:36.066908 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:36.067228 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:36.067958 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:07:36.067991 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:07:36.070719 (+ 2728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:07:36.074091 (+ 3372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:07:36.074099 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:07:36.074115 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:07:36.074612 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:07:36.075001 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:07:36.075004 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:07:36.075007 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:07:36.075200 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:07:36.075769 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:07:36.075775 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:07:36.076172 (+ 397us) client_negotiation.cc:769] Sending connection context
0926 15:07:36.076440 (+ 268us) client_negotiation.cc:240] Negotiation successful
0926 15:07:36.076889 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":271,"threads_started":1}
I20260926 15:07:36.079684 29268 heartbeater.cc:347] Connected to a master server at 127.9.123.126:37549
I20260926 15:07:36.079993 29268 heartbeater.cc:464] Registering TS with master...
I20260926 15:07:36.080781 29268 heartbeater.cc:511] Master 127.9.123.126:37549 requested a full tablet report, sending...
I20260926 15:07:36.083184 29079 ts_manager.cc:194] Registered new tserver with Master: 8b5051b8df88410ba4c9ab19ceb63353 (127.9.123.65:35603)
I20260926 15:07:36.084795 29079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.65@KRBTEST.COM'} at 127.9.123.65:45649
WARNING: no policy specified for kudu/127.9.123.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
Entry for principal kudu/127.9.123.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab.
I20260926 15:07:36.117661 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:37549
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:40369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:07:36.442678 29274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:36.442960 29274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:36.443085 29274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:36.450553 29274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:36.450670 29274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:36.450820 29274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:07:36.460861 29274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:40369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.66.keytab
--principal=kudu/127.9.123.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:37549
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:36.462173 29274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:36.463486 29274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:36.476567 29282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:36.476567 29280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:36.476701 29279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:36.477288 29274 server_base.cc:1034] running on GCE node
I20260926 15:07:36.477885 29274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:36.478876 29274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:36.480119 29274 hybrid_clock.cc:648] HybridClock initialized: now 1790435256480056 us; error 50 us; skew 500 ppm
I20260926 15:07:36.480558 29274 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:36 dist-test-slave-7153 krb5kdc[29034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435256, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:36.483722 29274 init.cc:377] Logged in from keytab as kudu/127.9.123.66@KRBTEST.COM (short username kudu)
I20260926 15:07:36.485190 29274 webserver.cc:460] Webserver started at http://127.9.123.66:46647/ using document root <none> and password file <none>
I20260926 15:07:36.485880 29274 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:36.485988 29274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:36.486279 29274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:36.489351 29274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "ed7d8416d3554e0191c182d2d4dc8d23"
format_stamp: "Formatted at 2026-09-26 15:07:36 on dist-test-slave-7153"
server_key: "8808bb28304d43b752b574a34dd3333d"
server_key_iv: "d4c72f92e7627455979a1960457fad76"
server_key_version: "encryptionkey@0"
I20260926 15:07:36.490156 29274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "ed7d8416d3554e0191c182d2d4dc8d23"
format_stamp: "Formatted at 2026-09-26 15:07:36 on dist-test-slave-7153"
server_key: "8808bb28304d43b752b574a34dd3333d"
server_key_iv: "d4c72f92e7627455979a1960457fad76"
server_key_version: "encryptionkey@0"
I20260926 15:07:36.495260 29274 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 15:07:36.499060 29289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:36.500599 29274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 15:07:36.500752 29274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "ed7d8416d3554e0191c182d2d4dc8d23"
format_stamp: "Formatted at 2026-09-26 15:07:36 on dist-test-slave-7153"
server_key: "8808bb28304d43b752b574a34dd3333d"
server_key_iv: "d4c72f92e7627455979a1960457fad76"
server_key_version: "encryptionkey@0"
I20260926 15:07:36.500953 29274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:36.534776 29274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:36.539170 29274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:36.539423 29274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:36.540356 29274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:07:36.541775 29274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:07:36.541855 29274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:36.541947 29274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:07:36.541988 29274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:36.571722 29274 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:45427
I20260926 15:07:36.571769 29401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:45427 every 8 connection(s)
I20260926 15:07:36.573027 29274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:07:36.581835 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 29274
I20260926 15:07:36.582008 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:07:36.582464 9709 external_mini_cluster.cc:1428] Setting key a22291021a67699d789f5e8967f91917
Sep 26 15:07:36 dist-test-slave-7153 krb5kdc[29034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435256, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.66@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:07:36.593621 29270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:36.575639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:58631 (local address 127.9.123.126:37549)
0926 15:07:36.575860 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:36.575872 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:36.577080 (+ 1208us) server_negotiation.cc:407] Connection header received
0926 15:07:36.579040 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:36.579046 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:36.579132 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:36.579222 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:36.580803 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:36.581604 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:36.583290 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:36.583947 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:36.584080 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:36.587646 (+ 3566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:36.587651 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:36.587655 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:36.587691 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:36.590302 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:36.591086 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:36.591091 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:36.591094 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:36.591230 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:36.591793 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:36.591797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:36.591799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:36.592104 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:36.592230 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:36.593145 (+ 915us) server_negotiation.cc:299] Negotiation successful
0926 15:07:36.593361 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":89}
I20260926 15:07:36.594957 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:36.576133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:37549 (local address 127.9.123.66:58631)
0926 15:07:36.576882 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:36.576930 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:36.578733 (+ 1803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:36.579497 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:36.579506 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:36.580026 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:07:36.580624 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:36.580643 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:36.581743 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:36.581747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:36.583130 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:36.583148 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:36.584171 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:36.584180 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:36.584457 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:36.585180 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:07:36.585210 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:07:36.587404 (+ 2194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:07:36.590470 (+ 3066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:07:36.590480 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:07:36.590499 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:07:36.590954 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:07:36.591340 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:07:36.591346 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:07:36.591350 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:07:36.591633 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:07:36.592230 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:07:36.592238 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:07:36.592713 (+ 475us) client_negotiation.cc:769] Sending connection context
0926 15:07:36.593013 (+ 300us) client_negotiation.cc:240] Negotiation successful
0926 15:07:36.593605 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":314,"threads_started":1}
I20260926 15:07:36.596726 29402 heartbeater.cc:347] Connected to a master server at 127.9.123.126:37549
I20260926 15:07:36.597067 29402 heartbeater.cc:464] Registering TS with master...
I20260926 15:07:36.597869 29402 heartbeater.cc:511] Master 127.9.123.126:37549 requested a full tablet report, sending...
I20260926 15:07:36.599473 29079 ts_manager.cc:194] Registered new tserver with Master: ed7d8416d3554e0191c182d2d4dc8d23 (127.9.123.66:45427)
I20260926 15:07:36.600526 29079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.66@KRBTEST.COM'} at 127.9.123.66:58631
WARNING: no policy specified for kudu/127.9.123.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
Entry for principal kudu/127.9.123.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab.
I20260926 15:07:36.637769 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:37549
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:40369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:07:36.962137 29407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:36.962450 29407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:36.962585 29407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:36.970268 29407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:36.970417 29407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:36.970597 29407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:07:36.980479 29407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:40369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.67.keytab
--principal=kudu/127.9.123.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:37549
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:36.981848 29407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:36.983273 29407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:36.997291 29415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:36.997377 29412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:36.997334 29413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:36.998131 29407 server_base.cc:1034] running on GCE node
I20260926 15:07:36.998766 29407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:36.999773 29407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:37.000986 29407 hybrid_clock.cc:648] HybridClock initialized: now 1790435257000927 us; error 37 us; skew 500 ppm
I20260926 15:07:37.001329 29407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:36 dist-test-slave-7153 krb5kdc[29034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435256, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:37.004335 29407 init.cc:377] Logged in from keytab as kudu/127.9.123.67@KRBTEST.COM (short username kudu)
I20260926 15:07:37.005957 29407 webserver.cc:460] Webserver started at http://127.9.123.67:45289/ using document root <none> and password file <none>
I20260926 15:07:37.006683 29407 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:37.006810 29407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:37.007113 29407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:37.010571 29407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "e08512f3b4904ae4bf7d7900124b212e"
format_stamp: "Formatted at 2026-09-26 15:07:37 on dist-test-slave-7153"
server_key: "beb7140ff7f57055820ab238e88987fe"
server_key_iv: "9e24970c1f3ecb0fea7c96574727fef7"
server_key_version: "encryptionkey@0"
I20260926 15:07:37.011471 29407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "e08512f3b4904ae4bf7d7900124b212e"
format_stamp: "Formatted at 2026-09-26 15:07:37 on dist-test-slave-7153"
server_key: "beb7140ff7f57055820ab238e88987fe"
server_key_iv: "9e24970c1f3ecb0fea7c96574727fef7"
server_key_version: "encryptionkey@0"
I20260926 15:07:37.017184 29407 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 15:07:37.021157 29422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:37.022897 29407 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260926 15:07:37.023105 29407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "e08512f3b4904ae4bf7d7900124b212e"
format_stamp: "Formatted at 2026-09-26 15:07:37 on dist-test-slave-7153"
server_key: "beb7140ff7f57055820ab238e88987fe"
server_key_iv: "9e24970c1f3ecb0fea7c96574727fef7"
server_key_version: "encryptionkey@0"
I20260926 15:07:37.023331 29407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:37.053894 29407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:37.059928 29407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:37.060266 29407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:37.061345 29407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:07:37.062716 29407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:07:37.062798 29407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:37.062911 29407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:07:37.062968 29407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:37.088201 29268 heartbeater.cc:503] Master 127.9.123.126:37549 was elected leader, sending a full tablet report...
I20260926 15:07:37.093833 29407 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:39883
I20260926 15:07:37.093914 29534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:39883 every 8 connection(s)
I20260926 15:07:37.095127 29407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:07:37.101599 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 29407
I20260926 15:07:37.101763 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:07:37.102159 9709 external_mini_cluster.cc:1428] Setting key 949d3e25dddf5a7fa8209812c2a3add4
Sep 26 15:07:37 dist-test-slave-7153 krb5kdc[29034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435256, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.67@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:07:37.116276 29537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:37.097639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:50105 (local address 127.9.123.126:37549)
0926 15:07:37.098069 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:37.098074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:37.099097 (+ 1023us) server_negotiation.cc:407] Connection header received
0926 15:07:37.101018 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:37.101023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:37.101136 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:37.101249 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:37.102920 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.103842 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.105930 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.106348 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.106447 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:37.109989 (+ 3542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:37.109995 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:37.109998 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:37.110034 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:37.112657 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:37.113589 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:37.113594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:37.113597 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:37.113781 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:37.114300 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:37.114304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:37.114306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:37.114606 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:37.114738 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:37.115591 (+ 853us) server_negotiation.cc:299] Negotiation successful
0926 15:07:37.115864 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":219,"threads_started":1}
I20260926 15:07:37.117360 29538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:37.098189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:37549 (local address 127.9.123.67:50105)
0926 15:07:37.098922 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:37.098968 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:37.100708 (+ 1740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:37.101499 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:37.101508 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:37.102081 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:07:37.102708 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:37.102736 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.104077 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.104083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:37.105796 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:37.105811 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.106577 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.106583 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:37.106812 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:37.107491 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:07:37.107516 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:07:37.109770 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:07:37.112827 (+ 3057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:07:37.112836 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:07:37.112852 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:07:37.113445 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:07:37.113921 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:07:37.113925 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:07:37.113928 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:07:37.114143 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:07:37.114759 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:07:37.114768 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:07:37.115137 (+ 369us) client_negotiation.cc:769] Sending connection context
0926 15:07:37.115436 (+ 299us) client_negotiation.cc:240] Negotiation successful
0926 15:07:37.116013 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":288,"threads_started":1}
I20260926 15:07:37.118731 29535 heartbeater.cc:347] Connected to a master server at 127.9.123.126:37549
I20260926 15:07:37.119076 29535 heartbeater.cc:464] Registering TS with master...
I20260926 15:07:37.119902 29535 heartbeater.cc:511] Master 127.9.123.126:37549 requested a full tablet report, sending...
I20260926 15:07:37.121840 29079 ts_manager.cc:194] Registered new tserver with Master: e08512f3b4904ae4bf7d7900124b212e (127.9.123.67:39883)
I20260926 15:07:37.122728 29079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.123.67@KRBTEST.COM'} at 127.9.123.67:50105
I20260926 15:07:37.130245 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 15:07:37.148702 29537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:37.134548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41432 (local address 127.9.123.126:37549)
0926 15:07:37.134716 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:37.134721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:37.135250 (+ 529us) server_negotiation.cc:407] Connection header received
0926 15:07:37.135468 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:37.135472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:37.135691 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:37.135859 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:37.137678 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.138459 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.140041 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.140503 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.140625 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:37.143171 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:37.143177 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:37.143181 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:37.143223 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:37.145973 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:37.146778 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:37.146783 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:37.146785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:37.146936 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:37.147297 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:37.147301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:37.147303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:37.147645 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:37.147831 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:37.148204 (+ 373us) server_negotiation.cc:299] Negotiation successful
0926 15:07:37.148465 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260926 15:07:37.152904 29079 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:07:37.160122 29079 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:07:37.199262 29551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:37.174470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:39883 (local address 127.0.0.1:41126)
0926 15:07:37.191352 (+ 16882us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:37.191367 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:37.191474 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:37.192133 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:37.192138 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:37.192162 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:37.192665 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:37.192679 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.194091 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.194097 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:37.196436 (+ 2339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:37.196448 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.198143 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.198150 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:37.198337 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:37.198392 (+ 55us) client_negotiation.cc:769] Sending connection context
0926 15:07:37.198555 (+ 163us) client_negotiation.cc:240] Negotiation successful
0926 15:07:37.198773 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16772,"mutex_wait_us":92,"thread_start_us":15988,"threads_started":1}
I20260926 15:07:37.199776 29553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:37.174803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41126 (local address 127.9.123.67:39883)
0926 15:07:37.191537 (+ 16734us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:37.191545 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:37.191575 (+ 30us) server_negotiation.cc:407] Connection header received
0926 15:07:37.191667 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:37.191674 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:37.191920 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:37.192119 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:37.192885 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.193960 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.196590 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.197927 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.198063 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:37.198316 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:37.198915 (+ 599us) server_negotiation.cc:299] Negotiation successful
0926 15:07:37.199124 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15035,"thread_start_us":14857,"threads_started":1}
I20260926 15:07:37.199790 29550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:37.173791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46456 (local address 127.9.123.66:45427)
0926 15:07:37.189282 (+ 15491us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:37.189291 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:37.191437 (+ 2146us) server_negotiation.cc:407] Connection header received
0926 15:07:37.191758 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:37.191766 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:37.192000 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:37.192189 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:37.192913 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.194654 (+ 1741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.197055 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.198070 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.198205 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:37.198434 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:37.199277 (+ 843us) server_negotiation.cc:299] Negotiation successful
0926 15:07:37.199454 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15240,"thread_start_us":15495,"threads_started":1}
I20260926 15:07:37.199925 29549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:37.173733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44528 (local address 127.9.123.65:35603)
0926 15:07:37.189568 (+ 15835us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:37.189576 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:37.190927 (+ 1351us) server_negotiation.cc:407] Connection header received
0926 15:07:37.191281 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:37.191290 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:37.191587 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:37.191779 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:37.192451 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.194173 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.197013 (+ 2840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.198431 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.198564 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:37.198791 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:37.199441 (+ 650us) server_negotiation.cc:299] Negotiation successful
0926 15:07:37.199622 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15678,"thread_start_us":15386,"threads_started":1}
I20260926 15:07:37.200059 29552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:37.174034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:35603 (local address 127.0.0.1:44528)
0926 15:07:37.190732 (+ 16698us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:37.190775 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:37.190971 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:37.191797 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:37.191805 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:37.191834 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:37.192240 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:37.192274 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.194328 (+ 2054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.194333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:37.196828 (+ 2495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:37.196846 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.198653 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.198660 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:37.198845 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:37.198877 (+ 32us) client_negotiation.cc:769] Sending connection context
0926 15:07:37.199010 (+ 133us) client_negotiation.cc:240] Negotiation successful
0926 15:07:37.199150 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16575,"mutex_wait_us":47,"thread_start_us":17057,"threads_started":1}
I20260926 15:07:37.200115 29548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:37.173519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:45427 (local address 127.0.0.1:46456)
0926 15:07:37.190550 (+ 17031us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:37.191320 (+ 770us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:37.191553 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:37.192387 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:37.192391 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:37.192419 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:37.192756 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:37.192772 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.194820 (+ 2048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.194825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:37.196933 (+ 2108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:37.196944 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.198591 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.198597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:37.198761 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:37.198790 (+ 29us) client_negotiation.cc:769] Sending connection context
0926 15:07:37.198874 (+ 84us) client_negotiation.cc:240] Negotiation successful
0926 15:07:37.199018 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16726,"thread_start_us":16592,"threads_started":1}
I20260926 15:07:37.204226 29337 tablet_service.cc:1467] Processing CreateTablet for tablet 2919ad205fb146cebf37c217fd5992ba (DEFAULT_TABLE table=test-table [id=006faa6ad9af469daad443191d4bd15a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:07:37.204327 29203 tablet_service.cc:1467] Processing CreateTablet for tablet 2919ad205fb146cebf37c217fd5992ba (DEFAULT_TABLE table=test-table [id=006faa6ad9af469daad443191d4bd15a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:07:37.204281 29470 tablet_service.cc:1467] Processing CreateTablet for tablet 2919ad205fb146cebf37c217fd5992ba (DEFAULT_TABLE table=test-table [id=006faa6ad9af469daad443191d4bd15a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:07:37.205790 29337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2919ad205fb146cebf37c217fd5992ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:37.205864 29203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2919ad205fb146cebf37c217fd5992ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:37.205890 29470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2919ad205fb146cebf37c217fd5992ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:37.219715 29554 tablet_bootstrap.cc:492] T 2919ad205fb146cebf37c217fd5992ba P ed7d8416d3554e0191c182d2d4dc8d23: Bootstrap starting.
I20260926 15:07:37.222054 29555 tablet_bootstrap.cc:492] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353: Bootstrap starting.
I20260926 15:07:37.223829 29554 tablet_bootstrap.cc:654] T 2919ad205fb146cebf37c217fd5992ba P ed7d8416d3554e0191c182d2d4dc8d23: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:37.223851 29556 tablet_bootstrap.cc:492] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e: Bootstrap starting.
I20260926 15:07:37.225384 29554 log.cc:824] T 2919ad205fb146cebf37c217fd5992ba P ed7d8416d3554e0191c182d2d4dc8d23: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:37.225839 29555 tablet_bootstrap.cc:654] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:37.227035 29555 log.cc:824] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:37.227214 29556 tablet_bootstrap.cc:654] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:37.228442 29556 log.cc:824] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:37.229579 29554 tablet_bootstrap.cc:492] T 2919ad205fb146cebf37c217fd5992ba P ed7d8416d3554e0191c182d2d4dc8d23: No bootstrap required, opened a new log
I20260926 15:07:37.229880 29554 ts_tablet_manager.cc:1397] T 2919ad205fb146cebf37c217fd5992ba P ed7d8416d3554e0191c182d2d4dc8d23: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260926 15:07:37.230122 29555 tablet_bootstrap.cc:492] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353: No bootstrap required, opened a new log
I20260926 15:07:37.230389 29555 ts_tablet_manager.cc:1397] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260926 15:07:37.235616 29556 tablet_bootstrap.cc:492] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e: No bootstrap required, opened a new log
I20260926 15:07:37.235926 29556 ts_tablet_manager.cc:1397] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260926 15:07:37.240983 29555 raft_consensus.cc:348] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed7d8416d3554e0191c182d2d4dc8d23" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45427 } } peers { permanent_uuid: "8b5051b8df88410ba4c9ab19ceb63353" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 35603 } } peers { permanent_uuid: "e08512f3b4904ae4bf7d7900124b212e" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 39883 } }
I20260926 15:07:37.240983 29554 raft_consensus.cc:348] T 2919ad205fb146cebf37c217fd5992ba P ed7d8416d3554e0191c182d2d4dc8d23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed7d8416d3554e0191c182d2d4dc8d23" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45427 } } peers { permanent_uuid: "8b5051b8df88410ba4c9ab19ceb63353" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 35603 } } peers { permanent_uuid: "e08512f3b4904ae4bf7d7900124b212e" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 39883 } }
I20260926 15:07:37.241552 29554 raft_consensus.cc:374] T 2919ad205fb146cebf37c217fd5992ba P ed7d8416d3554e0191c182d2d4dc8d23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:37.241549 29555 raft_consensus.cc:374] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:37.241662 29554 raft_consensus.cc:729] T 2919ad205fb146cebf37c217fd5992ba P ed7d8416d3554e0191c182d2d4dc8d23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed7d8416d3554e0191c182d2d4dc8d23, State: Initialized, Role: FOLLOWER
I20260926 15:07:37.241669 29555 raft_consensus.cc:729] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5051b8df88410ba4c9ab19ceb63353, State: Initialized, Role: FOLLOWER
I20260926 15:07:37.242404 29554 consensus_queue.cc:260] T 2919ad205fb146cebf37c217fd5992ba P ed7d8416d3554e0191c182d2d4dc8d23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed7d8416d3554e0191c182d2d4dc8d23" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45427 } } peers { permanent_uuid: "8b5051b8df88410ba4c9ab19ceb63353" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 35603 } } peers { permanent_uuid: "e08512f3b4904ae4bf7d7900124b212e" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 39883 } }
I20260926 15:07:37.242482 29555 consensus_queue.cc:260] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed7d8416d3554e0191c182d2d4dc8d23" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45427 } } peers { permanent_uuid: "8b5051b8df88410ba4c9ab19ceb63353" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 35603 } } peers { permanent_uuid: "e08512f3b4904ae4bf7d7900124b212e" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 39883 } }
I20260926 15:07:37.243932 29556 raft_consensus.cc:348] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed7d8416d3554e0191c182d2d4dc8d23" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45427 } } peers { permanent_uuid: "8b5051b8df88410ba4c9ab19ceb63353" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 35603 } } peers { permanent_uuid: "e08512f3b4904ae4bf7d7900124b212e" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 39883 } }
I20260926 15:07:37.244283 29402 heartbeater.cc:503] Master 127.9.123.126:37549 was elected leader, sending a full tablet report...
I20260926 15:07:37.244429 29556 raft_consensus.cc:374] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:37.244539 29556 raft_consensus.cc:729] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e08512f3b4904ae4bf7d7900124b212e, State: Initialized, Role: FOLLOWER
I20260926 15:07:37.245061 29554 ts_tablet_manager.cc:1428] T 2919ad205fb146cebf37c217fd5992ba P ed7d8416d3554e0191c182d2d4dc8d23: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260926 15:07:37.246013 29555 ts_tablet_manager.cc:1428] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260926 15:07:37.245980 29556 consensus_queue.cc:260] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed7d8416d3554e0191c182d2d4dc8d23" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45427 } } peers { permanent_uuid: "8b5051b8df88410ba4c9ab19ceb63353" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 35603 } } peers { permanent_uuid: "e08512f3b4904ae4bf7d7900124b212e" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 39883 } }
I20260926 15:07:37.247185 29535 heartbeater.cc:503] Master 127.9.123.126:37549 was elected leader, sending a full tablet report...
I20260926 15:07:37.247848 29556 ts_tablet_manager.cc:1428] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
W20260926 15:07:37.307296 29269 tablet.cc:2369] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 15:07:37.325258 29403 tablet.cc:2369] T 2919ad205fb146cebf37c217fd5992ba P ed7d8416d3554e0191c182d2d4dc8d23: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 15:07:37.346753 29536 tablet.cc:2369] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:07:37.348125 29561 raft_consensus.cc:482] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:07:37.348492 29562 raft_consensus.cc:482] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:07:37.348495 29561 raft_consensus.cc:504] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed7d8416d3554e0191c182d2d4dc8d23" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45427 } } peers { permanent_uuid: "8b5051b8df88410ba4c9ab19ceb63353" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 35603 } } peers { permanent_uuid: "e08512f3b4904ae4bf7d7900124b212e" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 39883 } }
I20260926 15:07:37.348762 29562 raft_consensus.cc:504] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed7d8416d3554e0191c182d2d4dc8d23" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45427 } } peers { permanent_uuid: "8b5051b8df88410ba4c9ab19ceb63353" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 35603 } } peers { permanent_uuid: "e08512f3b4904ae4bf7d7900124b212e" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 39883 } }
I20260926 15:07:37.350693 29562 leader_election.cc:288] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed7d8416d3554e0191c182d2d4dc8d23 (127.9.123.66:45427), 8b5051b8df88410ba4c9ab19ceb63353 (127.9.123.65:35603)
I20260926 15:07:37.350693 29561 leader_election.cc:288] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed7d8416d3554e0191c182d2d4dc8d23 (127.9.123.66:45427), e08512f3b4904ae4bf7d7900124b212e (127.9.123.67:39883)
I20260926 15:07:37.359962 29538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:37.351109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:45427 (local address 127.9.123.67:35605)
0926 15:07:37.351392 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:37.351412 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:37.351550 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:37.352193 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:37.352202 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:37.352236 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:37.352459 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:37.352475 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.354196 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.354201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:37.356229 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:37.356246 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.358241 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.358248 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:37.359307 (+ 1059us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:37.359338 (+ 31us) client_negotiation.cc:769] Sending connection context
0926 15:07:37.359439 (+ 101us) client_negotiation.cc:240] Negotiation successful
0926 15:07:37.359612 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"mutex_wait_us":799}
I20260926 15:07:37.360690 29550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:37.351123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:35605 (local address 127.9.123.66:45427)
0926 15:07:37.351845 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:37.351852 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:37.351874 (+ 22us) server_negotiation.cc:407] Connection header received
0926 15:07:37.351957 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:37.351963 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:37.352041 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:37.352131 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:37.352963 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.354089 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.356442 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.357888 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.358056 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:37.358143 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:37.360234 (+ 2091us) server_negotiation.cc:299] Negotiation successful
0926 15:07:37.360381 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552}
I20260926 15:07:37.361778 29357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2919ad205fb146cebf37c217fd5992ba" candidate_uuid: "e08512f3b4904ae4bf7d7900124b212e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed7d8416d3554e0191c182d2d4dc8d23" is_pre_election: true
I20260926 15:07:37.362578 29563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:37.350938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:39883 (local address 127.9.123.65:58627)
0926 15:07:37.351764 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:37.351782 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:37.351945 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:37.353208 (+ 1263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:37.353213 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:37.353240 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:37.353542 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:37.353554 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.355036 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.355041 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:37.357375 (+ 2334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:37.357393 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.361310 (+ 3917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.361317 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:37.361575 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:37.361904 (+ 329us) client_negotiation.cc:769] Sending connection context
0926 15:07:37.361995 (+ 91us) client_negotiation.cc:240] Negotiation successful
0926 15:07:37.362149 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"mutex_wait_us":275,"thread_start_us":530,"threads_started":1}
I20260926 15:07:37.362483 29357 raft_consensus.cc:2454] T 2919ad205fb146cebf37c217fd5992ba P ed7d8416d3554e0191c182d2d4dc8d23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e08512f3b4904ae4bf7d7900124b212e in term 0.
I20260926 15:07:37.362928 29553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:37.352613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:58627 (local address 127.9.123.67:39883)
0926 15:07:37.352790 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:37.352796 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:37.352825 (+ 29us) server_negotiation.cc:407] Connection header received
0926 15:07:37.352881 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:37.352886 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:37.352946 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:37.353031 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:37.353697 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.354854 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.357578 (+ 2724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.358981 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.359136 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:37.359218 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:37.362522 (+ 3304us) server_negotiation.cc:299] Negotiation successful
0926 15:07:37.362679 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":63}
I20260926 15:07:37.363381 29426 leader_election.cc:302] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e08512f3b4904ae4bf7d7900124b212e, ed7d8416d3554e0191c182d2d4dc8d23; no voters:
I20260926 15:07:37.363889 29562 raft_consensus.cc:2781] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:07:37.364063 29562 raft_consensus.cc:482] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:07:37.364161 29562 raft_consensus.cc:3037] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:37.363891 29490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2919ad205fb146cebf37c217fd5992ba" candidate_uuid: "8b5051b8df88410ba4c9ab19ceb63353" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e08512f3b4904ae4bf7d7900124b212e" is_pre_election: true
I20260926 15:07:37.364665 29549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:37.351992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:37771 (local address 127.9.123.65:35603)
0926 15:07:37.352156 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:37.352162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:37.352562 (+ 400us) server_negotiation.cc:407] Connection header received
0926 15:07:37.352797 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:37.352802 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:37.352889 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:37.352948 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:37.353734 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.354892 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.359784 (+ 4892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.361654 (+ 1870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.361789 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:37.361859 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:37.364246 (+ 2387us) server_negotiation.cc:299] Negotiation successful
0926 15:07:37.364414 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":37}
I20260926 15:07:37.365453 29564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:37.351406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:35603 (local address 127.9.123.67:37771)
0926 15:07:37.352483 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:37.352499 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:37.352649 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:37.353159 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:37.353164 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:37.353191 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:37.353543 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:37.353553 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.355083 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.355089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:37.359559 (+ 4470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:37.359576 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.363763 (+ 4187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.363770 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:37.363979 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:37.364050 (+ 71us) client_negotiation.cc:769] Sending connection context
0926 15:07:37.364965 (+ 915us) client_negotiation.cc:240] Negotiation successful
0926 15:07:37.365132 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":953,"mutex_wait_us":2305,"thread_start_us":756,"threads_started":1}
I20260926 15:07:37.366320 29223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2919ad205fb146cebf37c217fd5992ba" candidate_uuid: "e08512f3b4904ae4bf7d7900124b212e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5051b8df88410ba4c9ab19ceb63353" is_pre_election: true
I20260926 15:07:37.366748 29223 raft_consensus.cc:2454] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e08512f3b4904ae4bf7d7900124b212e in term 0.
I20260926 15:07:37.367467 29562 raft_consensus.cc:504] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed7d8416d3554e0191c182d2d4dc8d23" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45427 } } peers { permanent_uuid: "8b5051b8df88410ba4c9ab19ceb63353" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 35603 } } peers { permanent_uuid: "e08512f3b4904ae4bf7d7900124b212e" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 39883 } }
I20260926 15:07:37.367949 29490 raft_consensus.cc:2379] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8b5051b8df88410ba4c9ab19ceb63353 in current term 1: Already voted for candidate e08512f3b4904ae4bf7d7900124b212e in this term.
I20260926 15:07:37.368717 29223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2919ad205fb146cebf37c217fd5992ba" candidate_uuid: "e08512f3b4904ae4bf7d7900124b212e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b5051b8df88410ba4c9ab19ceb63353"
I20260926 15:07:37.368968 29223 raft_consensus.cc:3037] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:37.369560 29550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:37.352772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:35611 (local address 127.9.123.66:45427)
0926 15:07:37.361049 (+ 8277us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:37.361054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:37.362546 (+ 1492us) server_negotiation.cc:407] Connection header received
0926 15:07:37.362810 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:37.362829 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:37.362933 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:37.362997 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:37.363723 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.364763 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.367301 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.368428 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.368588 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:37.368641 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:37.369146 (+ 505us) server_negotiation.cc:299] Negotiation successful
0926 15:07:37.369287 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8169,"thread_start_us":10170,"threads_started":1}
I20260926 15:07:37.369621 29565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:37.352647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:45427 (local address 127.9.123.65:35611)
0926 15:07:37.362448 (+ 9801us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:37.362470 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:37.362662 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:37.363188 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:37.363193 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:37.363219 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:37.363557 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:37.363571 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.364965 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.364971 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:37.367070 (+ 2099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:37.367086 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.368780 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.368785 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:37.368946 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:37.368986 (+ 40us) client_negotiation.cc:769] Sending connection context
0926 15:07:37.369069 (+ 83us) client_negotiation.cc:240] Negotiation successful
0926 15:07:37.369255 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9668,"thread_start_us":9510,"threads_started":1}
I20260926 15:07:37.370088 29357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2919ad205fb146cebf37c217fd5992ba" candidate_uuid: "e08512f3b4904ae4bf7d7900124b212e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed7d8416d3554e0191c182d2d4dc8d23"
I20260926 15:07:37.370321 29562 leader_election.cc:288] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e [CANDIDATE]: Term 1 election: Requested vote from peers ed7d8416d3554e0191c182d2d4dc8d23 (127.9.123.66:45427), 8b5051b8df88410ba4c9ab19ceb63353 (127.9.123.65:35603)
I20260926 15:07:37.370318 29357 raft_consensus.cc:3037] T 2919ad205fb146cebf37c217fd5992ba P ed7d8416d3554e0191c182d2d4dc8d23 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:37.370479 29356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2919ad205fb146cebf37c217fd5992ba" candidate_uuid: "8b5051b8df88410ba4c9ab19ceb63353" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed7d8416d3554e0191c182d2d4dc8d23" is_pre_election: true
I20260926 15:07:37.371538 29159 leader_election.cc:302] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8b5051b8df88410ba4c9ab19ceb63353; no voters: e08512f3b4904ae4bf7d7900124b212e, ed7d8416d3554e0191c182d2d4dc8d23
I20260926 15:07:37.372288 29223 raft_consensus.cc:2454] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e08512f3b4904ae4bf7d7900124b212e in term 1.
I20260926 15:07:37.372462 29561 raft_consensus.cc:2726] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260926 15:07:37.373072 29425 leader_election.cc:302] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b5051b8df88410ba4c9ab19ceb63353, e08512f3b4904ae4bf7d7900124b212e; no voters:
I20260926 15:07:37.373486 29357 raft_consensus.cc:2454] T 2919ad205fb146cebf37c217fd5992ba P ed7d8416d3554e0191c182d2d4dc8d23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e08512f3b4904ae4bf7d7900124b212e in term 1.
I20260926 15:07:37.373510 29562 raft_consensus.cc:2781] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:07:37.374063 29562 raft_consensus.cc:686] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e [term 1 LEADER]: Becoming Leader. State: Replica: e08512f3b4904ae4bf7d7900124b212e, State: Running, Role: LEADER
I20260926 15:07:37.374571 29562 consensus_queue.cc:237] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed7d8416d3554e0191c182d2d4dc8d23" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45427 } } peers { permanent_uuid: "8b5051b8df88410ba4c9ab19ceb63353" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 35603 } } peers { permanent_uuid: "e08512f3b4904ae4bf7d7900124b212e" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 39883 } }
I20260926 15:07:37.378975 29079 catalog_manager.cc:5495] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e reported cstate change: term changed from 0 to 1, leader changed from <none> to e08512f3b4904ae4bf7d7900124b212e (127.9.123.67). New cstate: current_term: 1 leader_uuid: "e08512f3b4904ae4bf7d7900124b212e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed7d8416d3554e0191c182d2d4dc8d23" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b5051b8df88410ba4c9ab19ceb63353" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 35603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e08512f3b4904ae4bf7d7900124b212e" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 39883 } health_report { overall_health: HEALTHY } } }
I20260926 15:07:37.431442 29553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:37.425820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41140 (local address 127.9.123.67:39883)
0926 15:07:37.426016 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:37.426022 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:37.426042 (+ 20us) server_negotiation.cc:407] Connection header received
0926 15:07:37.426178 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:37.426182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:37.426269 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:37.426393 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:07:37.426853 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.427630 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.429090 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.429594 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.429693 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:37.430213 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:07:37.430779 (+ 566us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:07:37.430893 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:37.431017 (+ 124us) server_negotiation.cc:299] Negotiation successful
0926 15:07:37.431098 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 15:07:37.440038 29357 raft_consensus.cc:1261] T 2919ad205fb146cebf37c217fd5992ba P ed7d8416d3554e0191c182d2d4dc8d23 [term 1 FOLLOWER]: Refusing update from remote peer e08512f3b4904ae4bf7d7900124b212e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:07:37.440248 29223 raft_consensus.cc:1261] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353 [term 1 FOLLOWER]: Refusing update from remote peer e08512f3b4904ae4bf7d7900124b212e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:07:37.440881 29562 consensus_queue.cc:1035] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed7d8416d3554e0191c182d2d4dc8d23" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:07:37.441236 29567 consensus_queue.cc:1035] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5051b8df88410ba4c9ab19ceb63353" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 35603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:07:37.453837 29570 mvcc.cc:205] Tried to move back new op lower bound from 7333622814464147456 to 7333622814210252800. Current Snapshot: MvccSnapshot[applied={T|T < 7333622814464147456}]
I20260926 15:07:37.469997 29078 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:07:37.470353 29078 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:07:37.470753 29572 mvcc.cc:205] Tried to move back new op lower bound from 7333622814464147456 to 7333622814210252800. Current Snapshot: MvccSnapshot[applied={T|T < 7333622814464147456}]
I20260926 15:07:37.475198 29078 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 363e3edb0ded4aac91d4610f466e0bbb: Sending DeleteTablet for 3 replicas of tablet 2919ad205fb146cebf37c217fd5992ba
I20260926 15:07:37.476356 29337 tablet_service.cc:1514] Processing DeleteTablet for tablet 2919ad205fb146cebf37c217fd5992ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:07:37 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:46456
I20260926 15:07:37.476765 29203 tablet_service.cc:1514] Processing DeleteTablet for tablet 2919ad205fb146cebf37c217fd5992ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:07:37 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:44528
I20260926 15:07:37.477612 29470 tablet_service.cc:1514] Processing DeleteTablet for tablet 2919ad205fb146cebf37c217fd5992ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:07:37 UTC) from {username='kudu', principal='kudu/127.9.123.126@KRBTEST.COM'} at 127.0.0.1:41126
I20260926 15:07:37.478780 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 29140
I20260926 15:07:37.488255 29580 tablet_replica.cc:330] T 2919ad205fb146cebf37c217fd5992ba P ed7d8416d3554e0191c182d2d4dc8d23: stopping tablet replica
I20260926 15:07:37.488785 29580 raft_consensus.cc:2229] T 2919ad205fb146cebf37c217fd5992ba P ed7d8416d3554e0191c182d2d4dc8d23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:07:37.489442 29580 raft_consensus.cc:2258] T 2919ad205fb146cebf37c217fd5992ba P ed7d8416d3554e0191c182d2d4dc8d23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:07:37.490305 29581 tablet_replica.cc:330] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353: stopping tablet replica
I20260926 15:07:37.490629 29582 tablet_replica.cc:330] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e: stopping tablet replica
I20260926 15:07:37.490733 29581 raft_consensus.cc:2229] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:07:37.491117 29582 raft_consensus.cc:2229] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:07:37.491216 29581 raft_consensus.cc:2258] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:07:37.491874 29582 raft_consensus.cc:2258] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:07:37.493384 29580 ts_tablet_manager.cc:1905] T 2919ad205fb146cebf37c217fd5992ba P ed7d8416d3554e0191c182d2d4dc8d23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:07:37.494951 29581 ts_tablet_manager.cc:1905] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:07:37.500175 29582 ts_tablet_manager.cc:1905] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:07:37.500257 29580 ts_tablet_manager.cc:1918] T 2919ad205fb146cebf37c217fd5992ba P ed7d8416d3554e0191c182d2d4dc8d23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:07:37.500411 29580 log.cc:1196] T 2919ad205fb146cebf37c217fd5992ba P ed7d8416d3554e0191c182d2d4dc8d23: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/2919ad205fb146cebf37c217fd5992ba
Sep 26 15:07:37 dist-test-slave-7153 krb5kdc[29034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.65@KRBTEST.COM
I20260926 15:07:37.500924 29580 ts_tablet_manager.cc:1939] T 2919ad205fb146cebf37c217fd5992ba P ed7d8416d3554e0191c182d2d4dc8d23: Deleting consensus metadata
I20260926 15:07:37.502841 29066 catalog_manager.cc:4841] TS ed7d8416d3554e0191c182d2d4dc8d23 (127.9.123.66:45427): tablet 2919ad205fb146cebf37c217fd5992ba (table test-table [id=006faa6ad9af469daad443191d4bd15a]) successfully deleted
I20260926 15:07:37.506002 29581 ts_tablet_manager.cc:1918] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:07:37.506170 29581 log.cc:1196] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-0/wal/wals/2919ad205fb146cebf37c217fd5992ba
I20260926 15:07:37.506690 29581 ts_tablet_manager.cc:1939] T 2919ad205fb146cebf37c217fd5992ba P 8b5051b8df88410ba4c9ab19ceb63353: Deleting consensus metadata
I20260926 15:07:37.507972 29582 ts_tablet_manager.cc:1918] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:07:37.508148 29582 log.cc:1196] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/2919ad205fb146cebf37c217fd5992ba
I20260926 15:07:37.508538 29065 catalog_manager.cc:4841] TS 8b5051b8df88410ba4c9ab19ceb63353 (127.9.123.65:35603): tablet 2919ad205fb146cebf37c217fd5992ba (table test-table [id=006faa6ad9af469daad443191d4bd15a]) successfully deleted
I20260926 15:07:37.508679 29582 ts_tablet_manager.cc:1939] T 2919ad205fb146cebf37c217fd5992ba P e08512f3b4904ae4bf7d7900124b212e: Deleting consensus metadata
I20260926 15:07:37.510394 29549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:37.480260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44538 (local address 127.9.123.65:35603)
0926 15:07:37.480452 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:37.480459 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:37.486042 (+ 5583us) server_negotiation.cc:407] Connection header received
0926 15:07:37.486239 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:37.486244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:37.486350 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:37.486461 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:37.492525 (+ 6064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.493690 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.495636 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.496281 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.496399 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:37.502071 (+ 5672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:37.502088 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:37.502107 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:37.502153 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:37.506502 (+ 4349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:37.507470 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:37.507479 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:37.507483 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:37.507653 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:37.508190 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:37.508196 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:37.508200 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:37.508841 (+ 641us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:37.509169 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:37.509574 (+ 405us) server_negotiation.cc:299] Negotiation successful
0926 15:07:37.509814 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 15:07:37.510461 29063 catalog_manager.cc:4841] TS e08512f3b4904ae4bf7d7900124b212e (127.9.123.67:39883): tablet 2919ad205fb146cebf37c217fd5992ba (table test-table [id=006faa6ad9af469daad443191d4bd15a]) successfully deleted
I20260926 15:07:37.511348 29263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:37.604151 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 29140
I20260926 15:07:37.619168 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 29274
Sep 26 15:07:37 dist-test-slave-7153 krb5kdc[29034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.66@KRBTEST.COM
I20260926 15:07:37.635023 29550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:37.619971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46464 (local address 127.9.123.66:45427)
0926 15:07:37.620149 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:37.620161 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:37.620182 (+ 21us) server_negotiation.cc:407] Connection header received
0926 15:07:37.620408 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:37.620414 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:37.620484 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:37.620567 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:37.621899 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.622627 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.623901 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.624317 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.624423 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:37.629288 (+ 4865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:37.629306 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:37.629323 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:37.629369 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:37.632160 (+ 2791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:37.632864 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:37.632872 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:37.632874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:37.633010 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:37.633459 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:37.633463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:37.633465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:37.633847 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:37.634141 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:37.634493 (+ 352us) server_negotiation.cc:299] Negotiation successful
0926 15:07:37.634729 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 15:07:37.635722 29397 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:37.725456 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 29274
I20260926 15:07:37.739601 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 29407
Sep 26 15:07:37 dist-test-slave-7153 krb5kdc[29034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.67@KRBTEST.COM
I20260926 15:07:37.755951 29553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:37.740537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41144 (local address 127.9.123.67:39883)
0926 15:07:37.740775 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:37.740784 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:37.740805 (+ 21us) server_negotiation.cc:407] Connection header received
0926 15:07:37.740875 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:37.740878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:37.740944 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:37.741038 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:37.742378 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.743160 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.744452 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:37.744938 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:37.745052 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:37.749814 (+ 4762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:37.749828 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:37.749844 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:37.749882 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:37.753154 (+ 3272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:37.753886 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:37.753894 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:37.753897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:37.754040 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:37.754407 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:37.754410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:37.754412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:37.754794 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:37.755125 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:37.755400 (+ 275us) server_negotiation.cc:299] Negotiation successful
0926 15:07:37.755622 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 15:07:37.756592 29530 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:37.877858 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 29407
I20260926 15:07:37.893334 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 29047
I20260926 15:07:37.894457 29109 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:38.004168 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 29047
2026-09-26T15:07:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4278 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-26T15:07:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:07:38Z Disabled control of system clock
I20260926 15:07:38.075750 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:45445
--webserver_interface=127.9.123.126
--webserver_port=0
--builtin_ntp_servers=127.9.123.84:41539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:45445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 15:07:38.415788 29591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:38.416105 29591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:38.416186 29591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:38.423754 29591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:07:38.423874 29591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:38.423910 29591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:38.423965 29591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:07:38.424010 29591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:07:38.433429 29591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:41539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:45445
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:45445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:38.434700 29591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:38.436547 29591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:38.448503 29599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:38.448567 29597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:38.448604 29596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:38.449198 29591 server_base.cc:1034] running on GCE node
I20260926 15:07:38.450157 29591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:38.451637 29591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:38.452850 29591 hybrid_clock.cc:648] HybridClock initialized: now 1790435258452796 us; error 49 us; skew 500 ppm
I20260926 15:07:38.453264 29591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:07:38.455588 29591 webserver.cc:460] Webserver started at http://127.9.123.126:36735/ using document root <none> and password file <none>
I20260926 15:07:38.456308 29591 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:38.456416 29591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:38.456736 29591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:38.459890 29591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "07257c383707477ea40c8dba328fe6c6"
format_stamp: "Formatted at 2026-09-26 15:07:38 on dist-test-slave-7153"
server_key: "3202515fd432ffc9832d3ceab2a6a6fb"
server_key_iv: "bc930faa3a1c3c697d4c7dd1749b0b1c"
server_key_version: "encryptionkey@0"
I20260926 15:07:38.460784 29591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "07257c383707477ea40c8dba328fe6c6"
format_stamp: "Formatted at 2026-09-26 15:07:38 on dist-test-slave-7153"
server_key: "3202515fd432ffc9832d3ceab2a6a6fb"
server_key_iv: "bc930faa3a1c3c697d4c7dd1749b0b1c"
server_key_version: "encryptionkey@0"
I20260926 15:07:38.466393 29591 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 15:07:38.469997 29605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:38.471362 29591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:07:38.471560 29591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "07257c383707477ea40c8dba328fe6c6"
format_stamp: "Formatted at 2026-09-26 15:07:38 on dist-test-slave-7153"
server_key: "3202515fd432ffc9832d3ceab2a6a6fb"
server_key_iv: "bc930faa3a1c3c697d4c7dd1749b0b1c"
server_key_version: "encryptionkey@0"
I20260926 15:07:38.471796 29591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:38.519474 29591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:38.520416 29591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:38.520700 29591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:38.538331 29591 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:45445
I20260926 15:07:38.538349 29656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:45445 every 8 connection(s)
I20260926 15:07:38.539669 29591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:07:38.541054 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 29591
I20260926 15:07:38.541247 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:07:38.541785 9709 external_mini_cluster.cc:1428] Setting key 18287b75fe18d5e3a90716c0988c8cd1
I20260926 15:07:38.544538 29657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:38.555671 29660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:38.543754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44470 (local address 127.9.123.126:45445)
0926 15:07:38.544789 (+ 1035us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:38.544805 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:38.544850 (+ 45us) server_negotiation.cc:407] Connection header received
0926 15:07:38.546614 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:38.546636 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:38.547202 (+ 566us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:38.547633 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:38.548202 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:38.549690 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:38.551049 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:38.551721 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:38.551894 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:38.553211 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:38.553220 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:38.553236 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:38.553865 (+ 629us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:38.553969 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:38.553993 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:38.554103 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:38.554342 (+ 239us) server_negotiation.cc:299] Negotiation successful
0926 15:07:38.554685 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":333,"threads_started":1}
I20260926 15:07:38.556190 29657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07257c383707477ea40c8dba328fe6c6: Bootstrap starting.
I20260926 15:07:38.560034 29657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07257c383707477ea40c8dba328fe6c6: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:38.561169 29657 log.cc:824] T 00000000000000000000000000000000 P 07257c383707477ea40c8dba328fe6c6: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:38.564702 29657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07257c383707477ea40c8dba328fe6c6: No bootstrap required, opened a new log
I20260926 15:07:38.572642 29657 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07257c383707477ea40c8dba328fe6c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07257c383707477ea40c8dba328fe6c6" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45445 } }
I20260926 15:07:38.573071 29657 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07257c383707477ea40c8dba328fe6c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:38.573210 29657 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07257c383707477ea40c8dba328fe6c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07257c383707477ea40c8dba328fe6c6, State: Initialized, Role: FOLLOWER
I20260926 15:07:38.573822 29657 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07257c383707477ea40c8dba328fe6c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07257c383707477ea40c8dba328fe6c6" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45445 } }
I20260926 15:07:38.574049 29657 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07257c383707477ea40c8dba328fe6c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:07:38.574180 29657 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07257c383707477ea40c8dba328fe6c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:07:38.574353 29657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07257c383707477ea40c8dba328fe6c6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:38.576815 29657 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07257c383707477ea40c8dba328fe6c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07257c383707477ea40c8dba328fe6c6" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45445 } }
I20260926 15:07:38.577435 29657 leader_election.cc:302] T 00000000000000000000000000000000 P 07257c383707477ea40c8dba328fe6c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07257c383707477ea40c8dba328fe6c6; no voters:
I20260926 15:07:38.578029 29657 leader_election.cc:288] T 00000000000000000000000000000000 P 07257c383707477ea40c8dba328fe6c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:07:38.578168 29662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07257c383707477ea40c8dba328fe6c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:07:38.578805 29662 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07257c383707477ea40c8dba328fe6c6 [term 1 LEADER]: Becoming Leader. State: Replica: 07257c383707477ea40c8dba328fe6c6, State: Running, Role: LEADER
I20260926 15:07:38.579360 29662 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07257c383707477ea40c8dba328fe6c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07257c383707477ea40c8dba328fe6c6" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45445 } }
I20260926 15:07:38.580109 29657 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07257c383707477ea40c8dba328fe6c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:07:38.582245 29663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07257c383707477ea40c8dba328fe6c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07257c383707477ea40c8dba328fe6c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07257c383707477ea40c8dba328fe6c6" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45445 } } }
I20260926 15:07:38.582715 29663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07257c383707477ea40c8dba328fe6c6 [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:38.582216 29664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07257c383707477ea40c8dba328fe6c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07257c383707477ea40c8dba328fe6c6. Latest consensus state: current_term: 1 leader_uuid: "07257c383707477ea40c8dba328fe6c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07257c383707477ea40c8dba328fe6c6" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45445 } } }
I20260926 15:07:38.582944 29664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07257c383707477ea40c8dba328fe6c6 [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:38.588189 29667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:07:38.593719 29667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:07:38.602803 29667 catalog_manager.cc:1347] Generated new cluster ID: 9efda51041b643fab3471041ecd1dd9e
I20260926 15:07:38.602941 29667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:07:38.613632 29667 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:07:38.614938 29667 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:07:38.627171 29667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07257c383707477ea40c8dba328fe6c6: Generated new TSK 0
I20260926 15:07:38.628084 29667 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:07:38.634135 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.65:0
--local_ip_for_outbound_sockets=127.9.123.65
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:45445
--builtin_ntp_servers=127.9.123.84:41539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260926 15:07:39.001000 29681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:39.001343 29681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:39.001441 29681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:39.008872 29681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:39.008997 29681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:39.009160 29681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.65
I20260926 15:07:39.018611 29681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:41539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.123.65
--webserver_port=0
--tserver_master_addrs=127.9.123.126:45445
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.65
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:39.019886 29681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:39.021652 29681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:39.035320 29686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:39.035300 29687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:39.035367 29689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:39.036146 29681 server_base.cc:1034] running on GCE node
I20260926 15:07:39.036803 29681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:39.037868 29681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:39.039073 29681 hybrid_clock.cc:648] HybridClock initialized: now 1790435259039024 us; error 44 us; skew 500 ppm
I20260926 15:07:39.039402 29681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:07:39.041888 29681 webserver.cc:460] Webserver started at http://127.9.123.65:33175/ using document root <none> and password file <none>
I20260926 15:07:39.042634 29681 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:39.042733 29681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:39.043063 29681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:39.046257 29681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-0/data/instance:
uuid: "7ddf2fc75c384ff6ad4eff3abcfef002"
format_stamp: "Formatted at 2026-09-26 15:07:39 on dist-test-slave-7153"
server_key: "3488f906484a60f146a14fe78ba74e60"
server_key_iv: "5ffbf7d9e951fc55f2b9036a1b14cc45"
server_key_version: "encryptionkey@0"
I20260926 15:07:39.047089 29681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance:
uuid: "7ddf2fc75c384ff6ad4eff3abcfef002"
format_stamp: "Formatted at 2026-09-26 15:07:39 on dist-test-slave-7153"
server_key: "3488f906484a60f146a14fe78ba74e60"
server_key_iv: "5ffbf7d9e951fc55f2b9036a1b14cc45"
server_key_version: "encryptionkey@0"
I20260926 15:07:39.052848 29681 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 15:07:39.056690 29695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:39.058362 29681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:07:39.058569 29681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-0/wal
uuid: "7ddf2fc75c384ff6ad4eff3abcfef002"
format_stamp: "Formatted at 2026-09-26 15:07:39 on dist-test-slave-7153"
server_key: "3488f906484a60f146a14fe78ba74e60"
server_key_iv: "5ffbf7d9e951fc55f2b9036a1b14cc45"
server_key_version: "encryptionkey@0"
I20260926 15:07:39.058817 29681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:39.099260 29681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:39.100119 29681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:39.100414 29681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:39.101423 29681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:07:39.102963 29681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:07:39.103051 29681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:39.103137 29681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:07:39.103205 29681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:39.133942 29681 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.65:33207
I20260926 15:07:39.134004 29807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.65:33207 every 8 connection(s)
I20260926 15:07:39.135272 29681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-0/data/info.pb
I20260926 15:07:39.137018 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 29681
I20260926 15:07:39.137234 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-0/wal/instance
I20260926 15:07:39.137681 9709 external_mini_cluster.cc:1428] Setting key 1ea2d32c62604adb6c8b65cda18d644a
I20260926 15:07:39.140775 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.66:0
--local_ip_for_outbound_sockets=127.9.123.66
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:45445
--builtin_ntp_servers=127.9.123.84:41539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 15:07:39.149430 29810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:39.138024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.65:36857 (local address 127.9.123.126:45445)
0926 15:07:39.138473 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:39.138479 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:39.139915 (+ 1436us) server_negotiation.cc:407] Connection header received
0926 15:07:39.142361 (+ 2446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:39.142365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:39.142429 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:39.142521 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:39.143854 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:39.144535 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:39.146628 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:39.147029 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:39.147121 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:39.148629 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:39.148634 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:39.148637 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:39.148658 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:39.148698 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:39.148710 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:39.148790 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:39.149003 (+ 213us) server_negotiation.cc:299] Negotiation successful
0926 15:07:39.149085 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":214,"threads_started":1}
I20260926 15:07:39.150660 29811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:39.138674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45445 (local address 127.9.123.65:36857)
0926 15:07:39.139746 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:39.139798 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:39.142071 (+ 2273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:39.142769 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:39.142780 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:39.142909 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:07:39.143695 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:39.143719 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:39.144684 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:39.144689 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:39.146490 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:39.146507 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:39.147245 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:39.147251 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:39.147529 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:39.148265 (+ 736us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 15:07:39.148268 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:07:39.148322 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 15:07:39.148428 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:07:39.148804 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:07:39.148812 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:07:39.148820 (+ 8us) client_negotiation.cc:769] Sending connection context
0926 15:07:39.148998 (+ 178us) client_negotiation.cc:240] Negotiation successful
0926 15:07:39.149466 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":299,"threads_started":1}
I20260926 15:07:39.152228 29808 heartbeater.cc:347] Connected to a master server at 127.9.123.126:45445
I20260926 15:07:39.152540 29808 heartbeater.cc:464] Registering TS with master...
I20260926 15:07:39.153383 29808 heartbeater.cc:511] Master 127.9.123.126:45445 requested a full tablet report, sending...
I20260926 15:07:39.155627 29622 ts_manager.cc:194] Registered new tserver with Master: 7ddf2fc75c384ff6ad4eff3abcfef002 (127.9.123.65:33207)
I20260926 15:07:39.157605 29622 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.123.65:36857
W20260926 15:07:39.478998 29812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:39.479315 29812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:39.479406 29812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:39.486896 29812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:39.487011 29812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:39.487119 29812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.66
I20260926 15:07:39.496392 29812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:41539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.123.66
--webserver_port=0
--tserver_master_addrs=127.9.123.126:45445
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.66
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:39.497707 29812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:39.500200 29812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:39.519800 29818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:39.520013 29817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:39.519815 29820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:39.520785 29812 server_base.cc:1034] running on GCE node
I20260926 15:07:39.521458 29812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:39.522724 29812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:39.523975 29812 hybrid_clock.cc:648] HybridClock initialized: now 1790435259523872 us; error 68 us; skew 500 ppm
I20260926 15:07:39.524590 29812 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:07:39.528825 29812 webserver.cc:460] Webserver started at http://127.9.123.66:45755/ using document root <none> and password file <none>
I20260926 15:07:39.529883 29812 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:39.530053 29812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:39.530453 29812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:39.534016 29812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-1/data/instance:
uuid: "18a2dd3435cb41ccbb497954b5f89a95"
format_stamp: "Formatted at 2026-09-26 15:07:39 on dist-test-slave-7153"
server_key: "ccca975edc888c4d9ecc0eae9a90ac39"
server_key_iv: "072d39ce2467c1297f8f3c04e9ccc13e"
server_key_version: "encryptionkey@0"
I20260926 15:07:39.534929 29812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance:
uuid: "18a2dd3435cb41ccbb497954b5f89a95"
format_stamp: "Formatted at 2026-09-26 15:07:39 on dist-test-slave-7153"
server_key: "ccca975edc888c4d9ecc0eae9a90ac39"
server_key_iv: "072d39ce2467c1297f8f3c04e9ccc13e"
server_key_version: "encryptionkey@0"
I20260926 15:07:39.541796 29812 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260926 15:07:39.546773 29826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:39.548735 29812 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260926 15:07:39.549006 29812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-1/wal
uuid: "18a2dd3435cb41ccbb497954b5f89a95"
format_stamp: "Formatted at 2026-09-26 15:07:39 on dist-test-slave-7153"
server_key: "ccca975edc888c4d9ecc0eae9a90ac39"
server_key_iv: "072d39ce2467c1297f8f3c04e9ccc13e"
server_key_version: "encryptionkey@0"
I20260926 15:07:39.549428 29812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:39.562932 29812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:39.563872 29812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:39.564175 29812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:39.565320 29812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:07:39.567662 29812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:07:39.567780 29812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:39.567905 29812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:07:39.567977 29812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:39.633175 29812 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.66:45605
I20260926 15:07:39.633227 29938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.66:45605 every 8 connection(s)
I20260926 15:07:39.634565 29812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-1/data/info.pb
I20260926 15:07:39.639710 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 29812
I20260926 15:07:39.639897 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-1/wal/instance
I20260926 15:07:39.640430 9709 external_mini_cluster.cc:1428] Setting key e6e0bd74f6a2a667b4e62484b0ba8613
I20260926 15:07:39.644048 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.123.67:0
--local_ip_for_outbound_sockets=127.9.123.67
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:45445
--builtin_ntp_servers=127.9.123.84:41539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260926 15:07:39.649825 29810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:39.637273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:42027 (local address 127.9.123.126:45445)
0926 15:07:39.637483 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:39.637490 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:39.639078 (+ 1588us) server_negotiation.cc:407] Connection header received
0926 15:07:39.641500 (+ 2422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:39.641507 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:39.641602 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:39.641714 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:39.642895 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:39.643921 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:39.646381 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:39.646911 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:39.647029 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:39.648824 (+ 1795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:39.648831 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:39.648836 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:39.648864 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:39.648905 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:39.648921 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:39.649005 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:39.649411 (+ 406us) server_negotiation.cc:299] Negotiation successful
0926 15:07:39.649520 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 15:07:39.651270 29941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:39.637806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45445 (local address 127.9.123.66:42027)
0926 15:07:39.638887 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:39.638939 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:39.641161 (+ 2222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:39.641921 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:39.641930 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:39.642035 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:07:39.642736 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:39.642762 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:39.644103 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:39.644109 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:39.646208 (+ 2099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:39.646227 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:39.647161 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:39.647168 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:39.647501 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:39.648343 (+ 842us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 15:07:39.648347 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:07:39.648410 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 15:07:39.648539 (+ 129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:07:39.649016 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:07:39.649029 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:07:39.649038 (+ 9us) client_negotiation.cc:769] Sending connection context
0926 15:07:39.649399 (+ 361us) client_negotiation.cc:240] Negotiation successful
0926 15:07:39.649869 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":317,"threads_started":1}
I20260926 15:07:39.653087 29939 heartbeater.cc:347] Connected to a master server at 127.9.123.126:45445
I20260926 15:07:39.653479 29939 heartbeater.cc:464] Registering TS with master...
I20260926 15:07:39.654456 29939 heartbeater.cc:511] Master 127.9.123.126:45445 requested a full tablet report, sending...
I20260926 15:07:39.656162 29622 ts_manager.cc:194] Registered new tserver with Master: 18a2dd3435cb41ccbb497954b5f89a95 (127.9.123.66:45605)
I20260926 15:07:39.657212 29622 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.123.66:42027
W20260926 15:07:39.975401 29942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:39.975694 29942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:39.975788 29942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:39.983163 29942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:39.983274 29942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:39.983390 29942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.123.67
I20260926 15:07:39.992928 29942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:41539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.123.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.123.67
--webserver_port=0
--tserver_master_addrs=127.9.123.126:45445
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.123.67
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:39.994216 29942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:39.995954 29942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:40.008787 29947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:40.008760 29948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:40.008821 29950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:40.009739 29942 server_base.cc:1034] running on GCE node
I20260926 15:07:40.010366 29942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:40.011334 29942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:40.012562 29942 hybrid_clock.cc:648] HybridClock initialized: now 1790435260012514 us; error 43 us; skew 500 ppm
I20260926 15:07:40.012907 29942 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260926 15:07:40.015313 29942 webserver.cc:460] Webserver started at http://127.9.123.67:34865/ using document root <none> and password file <none>
I20260926 15:07:40.016016 29942 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:40.016120 29942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:40.016435 29942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:40.019668 29942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-2/data/instance:
uuid: "ccd37b1c773e417da54bcf570e4e5069"
format_stamp: "Formatted at 2026-09-26 15:07:40 on dist-test-slave-7153"
server_key: "c29d683a465f139542eb2c04e73783c6"
server_key_iv: "49e98b8ba72ceed169affbbda3a981b2"
server_key_version: "encryptionkey@0"
I20260926 15:07:40.020547 29942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance:
uuid: "ccd37b1c773e417da54bcf570e4e5069"
format_stamp: "Formatted at 2026-09-26 15:07:40 on dist-test-slave-7153"
server_key: "c29d683a465f139542eb2c04e73783c6"
server_key_iv: "49e98b8ba72ceed169affbbda3a981b2"
server_key_version: "encryptionkey@0"
I20260926 15:07:40.026304 29942 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 15:07:40.030144 29956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:40.032116 29942 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260926 15:07:40.032374 29942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-2/wal
uuid: "ccd37b1c773e417da54bcf570e4e5069"
format_stamp: "Formatted at 2026-09-26 15:07:40 on dist-test-slave-7153"
server_key: "c29d683a465f139542eb2c04e73783c6"
server_key_iv: "49e98b8ba72ceed169affbbda3a981b2"
server_key_version: "encryptionkey@0"
I20260926 15:07:40.032651 29942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:40.052181 29942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:40.053092 29942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:40.053457 29942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:40.054579 29942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260926 15:07:40.056469 29942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260926 15:07:40.056598 29942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:40.056725 29942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260926 15:07:40.056795 29942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:40.090787 29942 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.67:35199
I20260926 15:07:40.090816 30068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.67:35199 every 8 connection(s)
I20260926 15:07:40.092208 29942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-2/data/info.pb
I20260926 15:07:40.100991 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 29942
I20260926 15:07:40.101221 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-2/wal/instance
I20260926 15:07:40.101672 9709 external_mini_cluster.cc:1428] Setting key e8b742106c7539bf68c1062ecd1da9ec
I20260926 15:07:40.105847 29810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:40.094965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.67:57673 (local address 127.9.123.126:45445)
0926 15:07:40.095196 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:40.095203 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:40.096716 (+ 1513us) server_negotiation.cc:407] Connection header received
0926 15:07:40.098671 (+ 1955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:40.098676 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:40.098752 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:40.098851 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:40.100046 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.100823 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.102621 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.103208 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.103327 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:40.105027 (+ 1700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:40.105032 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:40.105036 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:40.105055 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:40.105090 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:40.105144 (+ 54us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:40.105235 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:40.105491 (+ 256us) server_negotiation.cc:299] Negotiation successful
0926 15:07:40.105578 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260926 15:07:40.106922 30071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:40.095518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:45445 (local address 127.9.123.67:57673)
0926 15:07:40.096505 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:40.096551 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:40.098320 (+ 1769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:40.099077 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:40.099087 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:40.099189 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:07:40.099848 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:40.099869 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.101019 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.101024 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:40.102467 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:40.102482 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.103478 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.103485 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:40.103776 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:40.104656 (+ 880us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0926 15:07:40.104659 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:07:40.104720 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0926 15:07:40.104812 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:07:40.105248 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:07:40.105258 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:07:40.105265 (+ 7us) client_negotiation.cc:769] Sending connection context
0926 15:07:40.105453 (+ 188us) client_negotiation.cc:240] Negotiation successful
0926 15:07:40.105820 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":291,"threads_started":1}
I20260926 15:07:40.108487 30069 heartbeater.cc:347] Connected to a master server at 127.9.123.126:45445
I20260926 15:07:40.108824 30069 heartbeater.cc:464] Registering TS with master...
I20260926 15:07:40.109761 30069 heartbeater.cc:511] Master 127.9.123.126:45445 requested a full tablet report, sending...
I20260926 15:07:40.111495 29622 ts_manager.cc:194] Registered new tserver with Master: ccd37b1c773e417da54bcf570e4e5069 (127.9.123.67:35199)
I20260926 15:07:40.112465 29622 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.123.67:57673
I20260926 15:07:40.117704 9709 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260926 15:07:40.128116 29810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:40.122353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35780 (local address 127.9.123.126:45445)
0926 15:07:40.122541 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:40.122546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:40.122856 (+ 310us) server_negotiation.cc:407] Connection header received
0926 15:07:40.123046 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:40.123051 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:40.123174 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:40.123283 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:40.123838 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.124595 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.126034 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.126473 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.126578 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:40.127553 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:40.127557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:40.127561 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:40.127582 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:40.127611 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:40.127622 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:40.127690 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:40.127793 (+ 103us) server_negotiation.cc:299] Negotiation successful
0926 15:07:40.127881 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 15:07:40.132848 29622 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35780:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260926 15:07:40.139791 29622 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260926 15:07:40.161181 29808 heartbeater.cc:503] Master 127.9.123.126:45445 was elected leader, sending a full tablet report...
I20260926 15:07:40.172570 30085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:40.156672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49068 (local address 127.9.123.65:33207)
0926 15:07:40.157293 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:40.157304 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:40.164765 (+ 7461us) server_negotiation.cc:407] Connection header received
0926 15:07:40.165134 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:40.165144 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:40.165409 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:40.165587 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:40.166396 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.167873 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.170325 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.171347 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.171482 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:40.171765 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:40.172064 (+ 299us) server_negotiation.cc:299] Negotiation successful
0926 15:07:40.172244 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":280,"threads_started":1}
I20260926 15:07:40.172674 30086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:40.156396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:33207 (local address 127.0.0.1:49068)
0926 15:07:40.164648 (+ 8252us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:40.164667 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:40.164891 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:40.165858 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:40.165863 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:40.165889 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:40.166220 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:40.166233 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.168023 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.168028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:40.170136 (+ 2108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:40.170146 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.171574 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.171580 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:40.171797 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:40.171854 (+ 57us) client_negotiation.cc:769] Sending connection context
0926 15:07:40.171979 (+ 125us) client_negotiation.cc:240] Negotiation successful
0926 15:07:40.172217 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8130,"mutex_wait_us":1,"spinlock_wait_cycles":14592,"thread_start_us":7990,"threads_started":1}
I20260926 15:07:40.173262 30081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:40.152531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:35199 (local address 127.0.0.1:53534)
0926 15:07:40.153434 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:40.153480 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:40.153674 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:40.164801 (+ 11127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:40.164809 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:40.164868 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:40.165338 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:40.165357 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.167264 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.167274 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:40.169708 (+ 2434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:40.169724 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.172013 (+ 2289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.172018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:40.172238 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:40.172306 (+ 68us) client_negotiation.cc:769] Sending connection context
0926 15:07:40.172386 (+ 80us) client_negotiation.cc:240] Negotiation successful
0926 15:07:40.172523 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"mutex_wait_us":44,"spinlock_wait_cycles":3328,"thread_start_us":1364,"threads_started":1}
I20260926 15:07:40.173349 30083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:40.152733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53534 (local address 127.9.123.67:35199)
0926 15:07:40.163907 (+ 11174us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:40.163917 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:40.163957 (+ 40us) server_negotiation.cc:407] Connection header received
0926 15:07:40.164076 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:40.164084 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:40.164505 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:40.165586 (+ 1081us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:40.165877 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.167142 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.169951 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.171450 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.171587 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:40.171863 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:40.172785 (+ 922us) server_negotiation.cc:299] Negotiation successful
0926 15:07:40.172963 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11008,"thread_start_us":10856,"threads_started":1}
I20260926 15:07:40.173919 30084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:40.153207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57016 (local address 127.9.123.66:45605)
0926 15:07:40.164246 (+ 11039us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:40.164255 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:40.164284 (+ 29us) server_negotiation.cc:407] Connection header received
0926 15:07:40.164381 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:40.164388 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:40.164634 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:40.164835 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:40.165739 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.167046 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.169924 (+ 2878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.171450 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.171587 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:40.171826 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:40.173410 (+ 1584us) server_negotiation.cc:299] Negotiation successful
0926 15:07:40.173612 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10882,"thread_start_us":10784,"threads_started":1}
I20260926 15:07:40.174446 30082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:40.152633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.66:45605 (local address 127.0.0.1:57016)
0926 15:07:40.154221 (+ 1588us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:40.154239 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:40.154355 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:40.165123 (+ 10768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:40.165128 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:40.165156 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:40.165534 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:40.165546 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.167190 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.167196 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:40.169667 (+ 2471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:40.169686 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.171993 (+ 2307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.172001 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:40.172243 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:40.172331 (+ 88us) client_negotiation.cc:769] Sending connection context
0926 15:07:40.172386 (+ 55us) client_negotiation.cc:240] Negotiation successful
0926 15:07:40.174206 (+ 1820us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1427,"mutex_wait_us":150,"thread_start_us":704,"threads_started":1}
I20260926 15:07:40.177587 30004 tablet_service.cc:1467] Processing CreateTablet for tablet e0a79635ecaf4de08bad12fefac72380 (DEFAULT_TABLE table=test-table [id=52c15dddf9ca43339f626f14c7f926d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:07:40.177734 29743 tablet_service.cc:1467] Processing CreateTablet for tablet e0a79635ecaf4de08bad12fefac72380 (DEFAULT_TABLE table=test-table [id=52c15dddf9ca43339f626f14c7f926d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:07:40.178002 29874 tablet_service.cc:1467] Processing CreateTablet for tablet e0a79635ecaf4de08bad12fefac72380 (DEFAULT_TABLE table=test-table [id=52c15dddf9ca43339f626f14c7f926d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260926 15:07:40.179025 30004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0a79635ecaf4de08bad12fefac72380. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:40.179143 29743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0a79635ecaf4de08bad12fefac72380. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:40.179311 29874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0a79635ecaf4de08bad12fefac72380. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:40.191336 30087 tablet_bootstrap.cc:492] T e0a79635ecaf4de08bad12fefac72380 P ccd37b1c773e417da54bcf570e4e5069: Bootstrap starting.
I20260926 15:07:40.194516 30088 tablet_bootstrap.cc:492] T e0a79635ecaf4de08bad12fefac72380 P 7ddf2fc75c384ff6ad4eff3abcfef002: Bootstrap starting.
I20260926 15:07:40.196154 30087 tablet_bootstrap.cc:654] T e0a79635ecaf4de08bad12fefac72380 P ccd37b1c773e417da54bcf570e4e5069: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:40.197526 30087 log.cc:824] T e0a79635ecaf4de08bad12fefac72380 P ccd37b1c773e417da54bcf570e4e5069: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:40.198264 30088 tablet_bootstrap.cc:654] T e0a79635ecaf4de08bad12fefac72380 P 7ddf2fc75c384ff6ad4eff3abcfef002: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:40.199106 30089 tablet_bootstrap.cc:492] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95: Bootstrap starting.
I20260926 15:07:40.199637 30088 log.cc:824] T e0a79635ecaf4de08bad12fefac72380 P 7ddf2fc75c384ff6ad4eff3abcfef002: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:40.201048 30087 tablet_bootstrap.cc:492] T e0a79635ecaf4de08bad12fefac72380 P ccd37b1c773e417da54bcf570e4e5069: No bootstrap required, opened a new log
I20260926 15:07:40.201395 30087 ts_tablet_manager.cc:1397] T e0a79635ecaf4de08bad12fefac72380 P ccd37b1c773e417da54bcf570e4e5069: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.002s
I20260926 15:07:40.203065 30088 tablet_bootstrap.cc:492] T e0a79635ecaf4de08bad12fefac72380 P 7ddf2fc75c384ff6ad4eff3abcfef002: No bootstrap required, opened a new log
I20260926 15:07:40.203428 30088 ts_tablet_manager.cc:1397] T e0a79635ecaf4de08bad12fefac72380 P 7ddf2fc75c384ff6ad4eff3abcfef002: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260926 15:07:40.203517 30089 tablet_bootstrap.cc:654] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:40.204617 30089 log.cc:824] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:40.207358 30089 tablet_bootstrap.cc:492] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95: No bootstrap required, opened a new log
I20260926 15:07:40.207638 30089 ts_tablet_manager.cc:1397] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.002s
I20260926 15:07:40.212276 30087 raft_consensus.cc:348] T e0a79635ecaf4de08bad12fefac72380 P ccd37b1c773e417da54bcf570e4e5069 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccd37b1c773e417da54bcf570e4e5069" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35199 } } peers { permanent_uuid: "18a2dd3435cb41ccbb497954b5f89a95" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45605 } } peers { permanent_uuid: "7ddf2fc75c384ff6ad4eff3abcfef002" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33207 } }
I20260926 15:07:40.212801 30087 raft_consensus.cc:374] T e0a79635ecaf4de08bad12fefac72380 P ccd37b1c773e417da54bcf570e4e5069 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:40.212968 30087 raft_consensus.cc:729] T e0a79635ecaf4de08bad12fefac72380 P ccd37b1c773e417da54bcf570e4e5069 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccd37b1c773e417da54bcf570e4e5069, State: Initialized, Role: FOLLOWER
I20260926 15:07:40.213716 30087 consensus_queue.cc:260] T e0a79635ecaf4de08bad12fefac72380 P ccd37b1c773e417da54bcf570e4e5069 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccd37b1c773e417da54bcf570e4e5069" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35199 } } peers { permanent_uuid: "18a2dd3435cb41ccbb497954b5f89a95" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45605 } } peers { permanent_uuid: "7ddf2fc75c384ff6ad4eff3abcfef002" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33207 } }
I20260926 15:07:40.213965 30088 raft_consensus.cc:348] T e0a79635ecaf4de08bad12fefac72380 P 7ddf2fc75c384ff6ad4eff3abcfef002 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccd37b1c773e417da54bcf570e4e5069" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35199 } } peers { permanent_uuid: "18a2dd3435cb41ccbb497954b5f89a95" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45605 } } peers { permanent_uuid: "7ddf2fc75c384ff6ad4eff3abcfef002" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33207 } }
I20260926 15:07:40.214510 30088 raft_consensus.cc:374] T e0a79635ecaf4de08bad12fefac72380 P 7ddf2fc75c384ff6ad4eff3abcfef002 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:40.214650 30088 raft_consensus.cc:729] T e0a79635ecaf4de08bad12fefac72380 P 7ddf2fc75c384ff6ad4eff3abcfef002 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ddf2fc75c384ff6ad4eff3abcfef002, State: Initialized, Role: FOLLOWER
I20260926 15:07:40.214905 30069 heartbeater.cc:503] Master 127.9.123.126:45445 was elected leader, sending a full tablet report...
I20260926 15:07:40.215453 30088 consensus_queue.cc:260] T e0a79635ecaf4de08bad12fefac72380 P 7ddf2fc75c384ff6ad4eff3abcfef002 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccd37b1c773e417da54bcf570e4e5069" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35199 } } peers { permanent_uuid: "18a2dd3435cb41ccbb497954b5f89a95" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45605 } } peers { permanent_uuid: "7ddf2fc75c384ff6ad4eff3abcfef002" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33207 } }
I20260926 15:07:40.215653 30087 ts_tablet_manager.cc:1428] T e0a79635ecaf4de08bad12fefac72380 P ccd37b1c773e417da54bcf570e4e5069: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s
I20260926 15:07:40.215811 30089 raft_consensus.cc:348] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccd37b1c773e417da54bcf570e4e5069" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35199 } } peers { permanent_uuid: "18a2dd3435cb41ccbb497954b5f89a95" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45605 } } peers { permanent_uuid: "7ddf2fc75c384ff6ad4eff3abcfef002" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33207 } }
I20260926 15:07:40.216235 30089 raft_consensus.cc:374] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:40.216331 30089 raft_consensus.cc:729] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18a2dd3435cb41ccbb497954b5f89a95, State: Initialized, Role: FOLLOWER
I20260926 15:07:40.216928 30089 consensus_queue.cc:260] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccd37b1c773e417da54bcf570e4e5069" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35199 } } peers { permanent_uuid: "18a2dd3435cb41ccbb497954b5f89a95" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45605 } } peers { permanent_uuid: "7ddf2fc75c384ff6ad4eff3abcfef002" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33207 } }
I20260926 15:07:40.217360 30088 ts_tablet_manager.cc:1428] T e0a79635ecaf4de08bad12fefac72380 P 7ddf2fc75c384ff6ad4eff3abcfef002: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260926 15:07:40.224732 29939 heartbeater.cc:503] Master 127.9.123.126:45445 was elected leader, sending a full tablet report...
I20260926 15:07:40.225525 30089 ts_tablet_manager.cc:1428] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95: Time spent starting tablet: real 0.018s user 0.017s sys 0.002s
W20260926 15:07:40.343926 30070 tablet.cc:2369] T e0a79635ecaf4de08bad12fefac72380 P ccd37b1c773e417da54bcf570e4e5069: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:07:40.348502 30095 raft_consensus.cc:482] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:07:40.348829 30095 raft_consensus.cc:504] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccd37b1c773e417da54bcf570e4e5069" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35199 } } peers { permanent_uuid: "18a2dd3435cb41ccbb497954b5f89a95" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45605 } } peers { permanent_uuid: "7ddf2fc75c384ff6ad4eff3abcfef002" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33207 } }
I20260926 15:07:40.350589 30095 leader_election.cc:288] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccd37b1c773e417da54bcf570e4e5069 (127.9.123.67:35199), 7ddf2fc75c384ff6ad4eff3abcfef002 (127.9.123.65:33207)
I20260926 15:07:40.357189 30083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:40.350969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:58637 (local address 127.9.123.67:35199)
0926 15:07:40.351200 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:40.351206 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:40.351564 (+ 358us) server_negotiation.cc:407] Connection header received
0926 15:07:40.351775 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:40.351781 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:40.351862 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:40.351949 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:40.352606 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.353544 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.355496 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.356320 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.356471 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:40.356542 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:40.356807 (+ 265us) server_negotiation.cc:299] Negotiation successful
0926 15:07:40.356926 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 15:07:40.357249 30096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:40.350861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.67:35199 (local address 127.9.123.66:58637)
0926 15:07:40.351431 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:40.351448 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:40.351613 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:40.352036 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:40.352039 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:40.352082 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:40.352462 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:40.352472 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.353660 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.353663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:40.355300 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:40.355317 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.356497 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.356503 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:40.356658 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:40.356682 (+ 24us) client_negotiation.cc:769] Sending connection context
0926 15:07:40.356761 (+ 79us) client_negotiation.cc:240] Negotiation successful
0926 15:07:40.356887 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":302,"threads_started":1}
I20260926 15:07:40.357729 30085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:40.351742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.123.66:52733 (local address 127.9.123.65:33207)
0926 15:07:40.351941 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:40.351946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:40.352255 (+ 309us) server_negotiation.cc:407] Connection header received
0926 15:07:40.352492 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:40.352497 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:40.352582 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:40.352683 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0926 15:07:40.353191 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.354084 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.356032 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.356818 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.356952 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:40.357017 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:40.357368 (+ 351us) server_negotiation.cc:299] Negotiation successful
0926 15:07:40.357489 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260926 15:07:40.358310 30097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:40.351532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.65:33207 (local address 127.9.123.66:52733)
0926 15:07:40.352143 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0926 15:07:40.352157 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0926 15:07:40.352298 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:07:40.352724 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:07:40.352728 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:07:40.352748 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0926 15:07:40.353021 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:40.353028 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.354267 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.354272 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:40.355846 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:07:40.355865 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.356989 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.356995 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:07:40.357190 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:40.357217 (+ 27us) client_negotiation.cc:769] Sending connection context
0926 15:07:40.357910 (+ 693us) client_negotiation.cc:240] Negotiation successful
0926 15:07:40.358051 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"mutex_wait_us":4,"thread_start_us":303,"threads_started":1}
I20260926 15:07:40.358426 30024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0a79635ecaf4de08bad12fefac72380" candidate_uuid: "18a2dd3435cb41ccbb497954b5f89a95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccd37b1c773e417da54bcf570e4e5069" is_pre_election: true
I20260926 15:07:40.358963 30024 raft_consensus.cc:2454] T e0a79635ecaf4de08bad12fefac72380 P ccd37b1c773e417da54bcf570e4e5069 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18a2dd3435cb41ccbb497954b5f89a95 in term 0.
I20260926 15:07:40.359081 29763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0a79635ecaf4de08bad12fefac72380" candidate_uuid: "18a2dd3435cb41ccbb497954b5f89a95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ddf2fc75c384ff6ad4eff3abcfef002" is_pre_election: true
I20260926 15:07:40.359551 29763 raft_consensus.cc:2454] T e0a79635ecaf4de08bad12fefac72380 P 7ddf2fc75c384ff6ad4eff3abcfef002 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18a2dd3435cb41ccbb497954b5f89a95 in term 0.
I20260926 15:07:40.359627 29830 leader_election.cc:302] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18a2dd3435cb41ccbb497954b5f89a95, ccd37b1c773e417da54bcf570e4e5069; no voters:
I20260926 15:07:40.360026 30095 raft_consensus.cc:2781] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260926 15:07:40.360253 30095 raft_consensus.cc:482] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260926 15:07:40.360368 30095 raft_consensus.cc:3037] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:40.362897 30095 raft_consensus.cc:504] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccd37b1c773e417da54bcf570e4e5069" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35199 } } peers { permanent_uuid: "18a2dd3435cb41ccbb497954b5f89a95" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45605 } } peers { permanent_uuid: "7ddf2fc75c384ff6ad4eff3abcfef002" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33207 } }
I20260926 15:07:40.363540 30095 leader_election.cc:288] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95 [CANDIDATE]: Term 1 election: Requested vote from peers ccd37b1c773e417da54bcf570e4e5069 (127.9.123.67:35199), 7ddf2fc75c384ff6ad4eff3abcfef002 (127.9.123.65:33207)
I20260926 15:07:40.364068 30024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0a79635ecaf4de08bad12fefac72380" candidate_uuid: "18a2dd3435cb41ccbb497954b5f89a95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccd37b1c773e417da54bcf570e4e5069"
I20260926 15:07:40.364136 29763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0a79635ecaf4de08bad12fefac72380" candidate_uuid: "18a2dd3435cb41ccbb497954b5f89a95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ddf2fc75c384ff6ad4eff3abcfef002"
I20260926 15:07:40.364320 30024 raft_consensus.cc:3037] T e0a79635ecaf4de08bad12fefac72380 P ccd37b1c773e417da54bcf570e4e5069 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:40.364329 29763 raft_consensus.cc:3037] T e0a79635ecaf4de08bad12fefac72380 P 7ddf2fc75c384ff6ad4eff3abcfef002 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:40.366546 30024 raft_consensus.cc:2454] T e0a79635ecaf4de08bad12fefac72380 P ccd37b1c773e417da54bcf570e4e5069 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18a2dd3435cb41ccbb497954b5f89a95 in term 1.
I20260926 15:07:40.366562 29763 raft_consensus.cc:2454] T e0a79635ecaf4de08bad12fefac72380 P 7ddf2fc75c384ff6ad4eff3abcfef002 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18a2dd3435cb41ccbb497954b5f89a95 in term 1.
I20260926 15:07:40.366983 29830 leader_election.cc:302] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18a2dd3435cb41ccbb497954b5f89a95, ccd37b1c773e417da54bcf570e4e5069; no voters:
I20260926 15:07:40.367308 30095 raft_consensus.cc:2781] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:07:40.367782 30095 raft_consensus.cc:686] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95 [term 1 LEADER]: Becoming Leader. State: Replica: 18a2dd3435cb41ccbb497954b5f89a95, State: Running, Role: LEADER
I20260926 15:07:40.368268 30095 consensus_queue.cc:237] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccd37b1c773e417da54bcf570e4e5069" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35199 } } peers { permanent_uuid: "18a2dd3435cb41ccbb497954b5f89a95" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45605 } } peers { permanent_uuid: "7ddf2fc75c384ff6ad4eff3abcfef002" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33207 } }
I20260926 15:07:40.373018 29622 catalog_manager.cc:5495] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18a2dd3435cb41ccbb497954b5f89a95 (127.9.123.66). New cstate: current_term: 1 leader_uuid: "18a2dd3435cb41ccbb497954b5f89a95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccd37b1c773e417da54bcf570e4e5069" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18a2dd3435cb41ccbb497954b5f89a95" member_type: VOTER last_known_addr { host: "127.9.123.66" port: 45605 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ddf2fc75c384ff6ad4eff3abcfef002" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33207 } health_report { overall_health: UNKNOWN } } }
W20260926 15:07:40.386863 29940 tablet.cc:2369] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260926 15:07:40.387570 29809 tablet.cc:2369] T e0a79635ecaf4de08bad12fefac72380 P 7ddf2fc75c384ff6ad4eff3abcfef002: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:07:40.419952 30084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:40.414267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57020 (local address 127.9.123.66:45605)
0926 15:07:40.414477 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:40.414482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:40.414526 (+ 44us) server_negotiation.cc:407] Connection header received
0926 15:07:40.414789 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:40.414794 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:40.414903 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:40.415040 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:07:40.415442 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.416246 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.417721 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.418149 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.418248 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:40.418735 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:07:40.419314 (+ 579us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:07:40.419432 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:40.419595 (+ 163us) server_negotiation.cc:299] Negotiation successful
0926 15:07:40.419669 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 15:07:40.428990 30024 raft_consensus.cc:1261] T e0a79635ecaf4de08bad12fefac72380 P ccd37b1c773e417da54bcf570e4e5069 [term 1 FOLLOWER]: Refusing update from remote peer 18a2dd3435cb41ccbb497954b5f89a95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:07:40.428992 29763 raft_consensus.cc:1261] T e0a79635ecaf4de08bad12fefac72380 P 7ddf2fc75c384ff6ad4eff3abcfef002 [term 1 FOLLOWER]: Refusing update from remote peer 18a2dd3435cb41ccbb497954b5f89a95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:07:40.430032 30095 consensus_queue.cc:1035] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ddf2fc75c384ff6ad4eff3abcfef002" member_type: VOTER last_known_addr { host: "127.9.123.65" port: 33207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:07:40.430470 30098 consensus_queue.cc:1035] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccd37b1c773e417da54bcf570e4e5069" member_type: VOTER last_known_addr { host: "127.9.123.67" port: 35199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:07:40.444360 30103 mvcc.cc:205] Tried to move back new op lower bound from 7333622826706444288 to 7333622826472742912. Current Snapshot: MvccSnapshot[applied={T|T < 7333622826706444288}]
I20260926 15:07:40.444664 30102 mvcc.cc:205] Tried to move back new op lower bound from 7333622826706444288 to 7333622826472742912. Current Snapshot: MvccSnapshot[applied={T|T < 7333622826706444288}]
I20260926 15:07:40.454936 29622 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:07:40.455230 29622 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260926 15:07:40.458945 29622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 07257c383707477ea40c8dba328fe6c6: Sending DeleteTablet for 3 replicas of tablet e0a79635ecaf4de08bad12fefac72380
I20260926 15:07:40.460227 29874 tablet_service.cc:1514] Processing DeleteTablet for tablet e0a79635ecaf4de08bad12fefac72380 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:07:40 UTC) from {username='slave'} at 127.0.0.1:57016
I20260926 15:07:40.460273 30004 tablet_service.cc:1514] Processing DeleteTablet for tablet e0a79635ecaf4de08bad12fefac72380 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:07:40 UTC) from {username='slave'} at 127.0.0.1:53534
I20260926 15:07:40.460459 29743 tablet_service.cc:1514] Processing DeleteTablet for tablet e0a79635ecaf4de08bad12fefac72380 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-26 15:07:40 UTC) from {username='slave'} at 127.0.0.1:49068
I20260926 15:07:40.462314 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 29681
I20260926 15:07:40.472754 30110 tablet_replica.cc:330] T e0a79635ecaf4de08bad12fefac72380 P ccd37b1c773e417da54bcf570e4e5069: stopping tablet replica
I20260926 15:07:40.473446 30110 raft_consensus.cc:2229] T e0a79635ecaf4de08bad12fefac72380 P ccd37b1c773e417da54bcf570e4e5069 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:07:40.474049 30110 raft_consensus.cc:2258] T e0a79635ecaf4de08bad12fefac72380 P ccd37b1c773e417da54bcf570e4e5069 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:07:40.476679 30110 ts_tablet_manager.cc:1905] T e0a79635ecaf4de08bad12fefac72380 P ccd37b1c773e417da54bcf570e4e5069: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:07:40.480754 30111 tablet_replica.cc:330] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95: stopping tablet replica
I20260926 15:07:40.481297 30111 raft_consensus.cc:2229] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95 [term 1 LEADER]: Raft consensus shutting down.
I20260926 15:07:40.482275 30111 raft_consensus.cc:2258] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:07:40.486902 30111 ts_tablet_manager.cc:1905] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:07:40.487365 30110 ts_tablet_manager.cc:1918] T e0a79635ecaf4de08bad12fefac72380 P ccd37b1c773e417da54bcf570e4e5069: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:07:40.487558 30110 log.cc:1196] T e0a79635ecaf4de08bad12fefac72380 P ccd37b1c773e417da54bcf570e4e5069: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-2/wal/wals/e0a79635ecaf4de08bad12fefac72380
I20260926 15:07:40.488095 30110 ts_tablet_manager.cc:1939] T e0a79635ecaf4de08bad12fefac72380 P ccd37b1c773e417da54bcf570e4e5069: Deleting consensus metadata
I20260926 15:07:40.490087 29609 catalog_manager.cc:4841] TS ccd37b1c773e417da54bcf570e4e5069 (127.9.123.67:35199): tablet e0a79635ecaf4de08bad12fefac72380 (table test-table [id=52c15dddf9ca43339f626f14c7f926d4]) successfully deleted
I20260926 15:07:40.491930 30112 tablet_replica.cc:330] T e0a79635ecaf4de08bad12fefac72380 P 7ddf2fc75c384ff6ad4eff3abcfef002: stopping tablet replica
I20260926 15:07:40.492317 30112 raft_consensus.cc:2229] T e0a79635ecaf4de08bad12fefac72380 P 7ddf2fc75c384ff6ad4eff3abcfef002 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260926 15:07:40.492746 30112 raft_consensus.cc:2258] T e0a79635ecaf4de08bad12fefac72380 P 7ddf2fc75c384ff6ad4eff3abcfef002 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260926 15:07:40.494777 30111 ts_tablet_manager.cc:1918] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:07:40.494963 30111 log.cc:1196] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-1/wal/wals/e0a79635ecaf4de08bad12fefac72380
I20260926 15:07:40.495061 30112 ts_tablet_manager.cc:1905] T e0a79635ecaf4de08bad12fefac72380 P 7ddf2fc75c384ff6ad4eff3abcfef002: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260926 15:07:40.495502 30111 ts_tablet_manager.cc:1939] T e0a79635ecaf4de08bad12fefac72380 P 18a2dd3435cb41ccbb497954b5f89a95: Deleting consensus metadata
I20260926 15:07:40.497287 29608 catalog_manager.cc:4841] TS 18a2dd3435cb41ccbb497954b5f89a95 (127.9.123.66:45605): tablet e0a79635ecaf4de08bad12fefac72380 (table test-table [id=52c15dddf9ca43339f626f14c7f926d4]) successfully deleted
I20260926 15:07:40.500070 30112 ts_tablet_manager.cc:1918] T e0a79635ecaf4de08bad12fefac72380 P 7ddf2fc75c384ff6ad4eff3abcfef002: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260926 15:07:40.500237 30112 log.cc:1196] T e0a79635ecaf4de08bad12fefac72380 P 7ddf2fc75c384ff6ad4eff3abcfef002: Deleting WAL directory at /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790434939095847-9709-0/minicluster-data/ts-0/wal/wals/e0a79635ecaf4de08bad12fefac72380
I20260926 15:07:40.500631 30112 ts_tablet_manager.cc:1939] T e0a79635ecaf4de08bad12fefac72380 P 7ddf2fc75c384ff6ad4eff3abcfef002: Deleting consensus metadata
I20260926 15:07:40.502192 29609 catalog_manager.cc:4841] TS 7ddf2fc75c384ff6ad4eff3abcfef002 (127.9.123.65:33207): tablet e0a79635ecaf4de08bad12fefac72380 (table test-table [id=52c15dddf9ca43339f626f14c7f926d4]) successfully deleted
I20260926 15:07:40.505923 30085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:40.463749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49078 (local address 127.9.123.65:33207)
0926 15:07:40.463927 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:40.463933 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:40.486404 (+ 22471us) server_negotiation.cc:407] Connection header received
0926 15:07:40.486884 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:40.486890 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:40.487028 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:40.487120 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:40.487817 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.488938 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.490741 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.491476 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.491620 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:40.504458 (+ 12838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:40.504467 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:40.504480 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:40.505132 (+ 652us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:40.505208 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:40.505234 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:40.505338 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:40.505496 (+ 158us) server_negotiation.cc:299] Negotiation successful
0926 15:07:40.505622 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260926 15:07:40.506635 29803 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:40.609452 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 29681
I20260926 15:07:40.632248 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 29812
I20260926 15:07:40.641515 30084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:40.633185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57022 (local address 127.9.123.66:45605)
0926 15:07:40.633390 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:40.633396 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:40.633421 (+ 25us) server_negotiation.cc:407] Connection header received
0926 15:07:40.633623 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:40.633628 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:40.633732 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:40.633847 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:40.634295 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.635528 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.637491 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.638255 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.638389 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:40.639526 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:40.639536 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:40.639554 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:40.640398 (+ 844us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:40.640463 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:40.640488 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:40.640596 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:40.640746 (+ 150us) server_negotiation.cc:299] Negotiation successful
0926 15:07:40.640856 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260926 15:07:40.642318 29934 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:40.754010 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 29812
I20260926 15:07:40.769239 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 29942
I20260926 15:07:40.776088 30083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:40.769956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53538 (local address 127.9.123.67:35199)
0926 15:07:40.770155 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:40.770160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:40.770180 (+ 20us) server_negotiation.cc:407] Connection header received
0926 15:07:40.770345 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:40.770350 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:40.770434 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:40.770537 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:40.770953 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.771707 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.773075 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:40.773559 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:40.773671 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:40.774671 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:40.774681 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:40.774695 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:40.775363 (+ 668us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:40.775414 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:40.775436 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:40.775529 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:40.775657 (+ 128us) server_negotiation.cc:299] Negotiation successful
0926 15:07:40.775768 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260926 15:07:40.776790 30064 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:40.864171 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 29942
I20260926 15:07:40.877734 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 29591
I20260926 15:07:40.878773 29652 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:40.986349 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 29591
2026-09-26T15:07:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2981 ms)
[----------] 6 tests from EncryptionPolicyTest (21476 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:07:41 dist-test-slave-7153 krb5kdc[30120](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:07:41 dist-test-slave-7153 krb5kdc[30120](info): set up 2 sockets
Sep 26 15:07:41 dist-test-slave-7153 krb5kdc[30120](info): commencing operation
krb5kdc: starting...
W20260926 15:07:42.017405 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.005s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:07:42 dist-test-slave-7153 krb5kdc[30120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:07:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:07:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 15:07:42.269423 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37029
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:07:42.595737 30133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:42.596055 30133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:42.596191 30133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:42.603513 30133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:07:42.603652 30133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:42.603705 30133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:42.603789 30133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:07:42.603845 30133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:07:42.613998 30133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37029
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:42.615324 30133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:42.616647 30133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:42.627960 30138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:42.627938 30139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:42.628140 30141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:42.628886 30133 server_base.cc:1034] running on GCE node
I20260926 15:07:42.629679 30133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:42.631059 30133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:42.632283 30133 hybrid_clock.cc:648] HybridClock initialized: now 1790435262632238 us; error 45 us; skew 500 ppm
I20260926 15:07:42.632651 30133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:42 dist-test-slave-7153 krb5kdc[30120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435262, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:42.635651 30133 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 15:07:42.637147 30133 webserver.cc:460] Webserver started at http://127.0.0.1:40447/ using document root <none> and password file <none>
I20260926 15:07:42.637850 30133 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:42.637972 30133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:42.638257 30133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:42.641335 30133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "35bf944b949548608237d0383c7d0707"
format_stamp: "Formatted at 2026-09-26 15:07:42 on dist-test-slave-7153"
server_key: "f4bc665d8f5c19146f9a719d9bbdc49c"
server_key_iv: "b85f94a9ac4b4179469c156e05ead8cb"
server_key_version: "encryptionkey@0"
I20260926 15:07:42.642100 30133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "35bf944b949548608237d0383c7d0707"
format_stamp: "Formatted at 2026-09-26 15:07:42 on dist-test-slave-7153"
server_key: "f4bc665d8f5c19146f9a719d9bbdc49c"
server_key_iv: "b85f94a9ac4b4179469c156e05ead8cb"
server_key_version: "encryptionkey@0"
I20260926 15:07:42.647307 30133 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 15:07:42.650831 30148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:42.652242 30133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 15:07:42.652401 30133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "35bf944b949548608237d0383c7d0707"
format_stamp: "Formatted at 2026-09-26 15:07:42 on dist-test-slave-7153"
server_key: "f4bc665d8f5c19146f9a719d9bbdc49c"
server_key_iv: "b85f94a9ac4b4179469c156e05ead8cb"
server_key_version: "encryptionkey@0"
I20260926 15:07:42.652595 30133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:42.672343 30133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:42.676518 30133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:42.676786 30133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:42.694065 30133 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37029
I20260926 15:07:42.694104 30199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37029 every 8 connection(s)
I20260926 15:07:42.695428 30133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:07:42.699149 30200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:42.703119 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 30133
I20260926 15:07:42.703364 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:07:42.704151 9709 external_mini_cluster.cc:1428] Setting key de964c77a576333e45b05bb7b197eeb6
I20260926 15:07:42.711602 30200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35bf944b949548608237d0383c7d0707: Bootstrap starting.
I20260926 15:07:42.715843 30200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35bf944b949548608237d0383c7d0707: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:42.717226 30200 log.cc:824] T 00000000000000000000000000000000 P 35bf944b949548608237d0383c7d0707: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:42.720909 30200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35bf944b949548608237d0383c7d0707: No bootstrap required, opened a new log
Sep 26 15:07:42 dist-test-slave-7153 krb5kdc[30120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 15:07:42.729962 30200 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35bf944b949548608237d0383c7d0707 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35bf944b949548608237d0383c7d0707" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37029 } }
I20260926 15:07:42.730360 30200 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35bf944b949548608237d0383c7d0707 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:42.730445 30203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:42.706311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43829 (local address 127.0.0.1:37029)
0926 15:07:42.707310 (+ 999us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:42.707326 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:42.707730 (+ 404us) server_negotiation.cc:407] Connection header received
0926 15:07:42.710053 (+ 2323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:42.710081 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:42.710532 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:42.710951 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:42.712759 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:42.714213 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:42.716354 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:42.717180 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:42.717340 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:42.722429 (+ 5089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:42.722437 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:42.722454 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:42.722496 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:42.725761 (+ 3265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:42.726792 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:42.726799 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:42.726807 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:42.726984 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:42.727496 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:42.727500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:42.727502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:42.728176 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:42.728454 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:42.728981 (+ 527us) server_negotiation.cc:299] Negotiation successful
0926 15:07:42.729353 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706,"thread_start_us":417,"threads_started":1}
I20260926 15:07:42.730461 30200 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35bf944b949548608237d0383c7d0707 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35bf944b949548608237d0383c7d0707, State: Initialized, Role: FOLLOWER
I20260926 15:07:42.731269 30200 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35bf944b949548608237d0383c7d0707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35bf944b949548608237d0383c7d0707" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37029 } }
I20260926 15:07:42.731570 30200 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35bf944b949548608237d0383c7d0707 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:07:42.731710 30200 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35bf944b949548608237d0383c7d0707 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:07:42.731897 30200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35bf944b949548608237d0383c7d0707 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:42.734841 30200 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35bf944b949548608237d0383c7d0707 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35bf944b949548608237d0383c7d0707" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37029 } }
I20260926 15:07:42.734997 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:07:42.735386 30200 leader_election.cc:302] T 00000000000000000000000000000000 P 35bf944b949548608237d0383c7d0707 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35bf944b949548608237d0383c7d0707; no voters:
I20260926 15:07:42.735975 30200 leader_election.cc:288] T 00000000000000000000000000000000 P 35bf944b949548608237d0383c7d0707 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:07:42.736130 30205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35bf944b949548608237d0383c7d0707 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:07:42.737919 30200 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35bf944b949548608237d0383c7d0707 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:07:42.742578 30205 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35bf944b949548608237d0383c7d0707 [term 1 LEADER]: Becoming Leader. State: Replica: 35bf944b949548608237d0383c7d0707, State: Running, Role: LEADER
I20260926 15:07:42.743446 30205 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35bf944b949548608237d0383c7d0707 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35bf944b949548608237d0383c7d0707" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37029 } }
I20260926 15:07:42.746562 30207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35bf944b949548608237d0383c7d0707 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35bf944b949548608237d0383c7d0707. Latest consensus state: current_term: 1 leader_uuid: "35bf944b949548608237d0383c7d0707" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35bf944b949548608237d0383c7d0707" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37029 } } }
I20260926 15:07:42.746927 30207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35bf944b949548608237d0383c7d0707 [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:42.746529 30206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35bf944b949548608237d0383c7d0707 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35bf944b949548608237d0383c7d0707" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35bf944b949548608237d0383c7d0707" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37029 } } }
I20260926 15:07:42.747561 30206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35bf944b949548608237d0383c7d0707 [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:42.762254 30203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:42.744921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33977 (local address 127.0.0.1:37029)
0926 15:07:42.745077 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:42.745082 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:42.745696 (+ 614us) server_negotiation.cc:407] Connection header received
0926 15:07:42.745899 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:42.745904 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:42.745988 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:42.746148 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:42.748072 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:42.749379 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:42.751366 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:42.752133 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:42.752230 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:42.755858 (+ 3628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:42.755866 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:42.755871 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:42.755917 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:42.759153 (+ 3236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:42.760065 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:42.760068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:42.760071 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:42.760197 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:42.760671 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:42.760676 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:42.760680 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:42.761019 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:42.761361 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:42.761721 (+ 360us) server_negotiation.cc:299] Negotiation successful
0926 15:07:42.761906 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260926 15:07:42.762969 30224 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 35bf944b949548608237d0383c7d0707: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 15:07:42.763101 30224 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 15:07:42.763273 30225 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:07:42.765041 30225 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:07:42.773435 30225 catalog_manager.cc:1347] Generated new cluster ID: 9afde210103b4f93813729105aee7307
I20260926 15:07:42.773542 30225 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:07:42.797858 30225 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:07:42.799700 30225 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:07:42.818001 30225 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35bf944b949548608237d0383c7d0707: Generated new TSK 0
I20260926 15:07:42.819025 30225 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:07:42.836038 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 30133
I20260926 15:07:42.837482 30195 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:42.933449 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 30133
W20260926 15:07:42.948423 30118 connection.cc:535] client connection to 127.0.0.1:37029 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37029): Connection reset by peer (error 104)
2026-09-26T15:07:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1942 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:07:43 dist-test-slave-7153 krb5kdc[30232](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:07:43 dist-test-slave-7153 krb5kdc[30232](info): set up 2 sockets
Sep 26 15:07:43 dist-test-slave-7153 krb5kdc[30232](info): commencing operation
krb5kdc: starting...
W20260926 15:07:43.949314 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.982s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:07:44 dist-test-slave-7153 krb5kdc[30232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:07:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:07:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 15:07:44.204304 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42799
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:07:44.541612 30245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:44.541961 30245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:44.542091 30245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:44.549960 30245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:07:44.550109 30245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:44.550196 30245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:44.550289 30245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:07:44.550344 30245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:07:44.560737 30245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42799
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:44.562191 30245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:44.563544 30245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:44.575646 30250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:44.575614 30253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:44.575578 30251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:44.576447 30245 server_base.cc:1034] running on GCE node
I20260926 15:07:44.577349 30245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:44.578910 30245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:44.580159 30245 hybrid_clock.cc:648] HybridClock initialized: now 1790435264580100 us; error 86 us; skew 500 ppm
I20260926 15:07:44.580566 30245 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:44 dist-test-slave-7153 krb5kdc[30232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435264, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:44.584010 30245 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 15:07:44.585638 30245 webserver.cc:460] Webserver started at http://127.0.0.1:39383/ using document root <none> and password file <none>
I20260926 15:07:44.586359 30245 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:44.586488 30245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:44.586778 30245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:44.589970 30245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "e3b70ed5442644138d8bb5141e04778b"
format_stamp: "Formatted at 2026-09-26 15:07:44 on dist-test-slave-7153"
server_key: "23205b0aed2c38661923b6cef0df90b6"
server_key_iv: "42577229637fc577d30a27f05139d423"
server_key_version: "encryptionkey@0"
I20260926 15:07:44.590741 30245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "e3b70ed5442644138d8bb5141e04778b"
format_stamp: "Formatted at 2026-09-26 15:07:44 on dist-test-slave-7153"
server_key: "23205b0aed2c38661923b6cef0df90b6"
server_key_iv: "42577229637fc577d30a27f05139d423"
server_key_version: "encryptionkey@0"
I20260926 15:07:44.596369 30245 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260926 15:07:44.600208 30260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:44.601745 30245 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 15:07:44.601931 30245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "e3b70ed5442644138d8bb5141e04778b"
format_stamp: "Formatted at 2026-09-26 15:07:44 on dist-test-slave-7153"
server_key: "23205b0aed2c38661923b6cef0df90b6"
server_key_iv: "42577229637fc577d30a27f05139d423"
server_key_version: "encryptionkey@0"
I20260926 15:07:44.602141 30245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:44.628803 30245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:44.632853 30245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:44.633157 30245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:44.650283 30245 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42799
I20260926 15:07:44.650326 30311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42799 every 8 connection(s)
I20260926 15:07:44.651674 30245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:07:44.655462 30312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:44.658764 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 30245
I20260926 15:07:44.658923 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:07:44.659351 9709 external_mini_cluster.cc:1428] Setting key 090a7120c706124c33099ce4daf5ba9c
I20260926 15:07:44.667654 30312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3b70ed5442644138d8bb5141e04778b: Bootstrap starting.
I20260926 15:07:44.671248 30312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3b70ed5442644138d8bb5141e04778b: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:44.672427 30312 log.cc:824] T 00000000000000000000000000000000 P e3b70ed5442644138d8bb5141e04778b: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:44.672598 30315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:44.661225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33045 (local address 127.0.0.1:42799)
0926 15:07:44.662004 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:44.662016 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:44.662057 (+ 41us) server_negotiation.cc:407] Connection header received
0926 15:07:44.663689 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:44.663712 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:44.664145 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:44.664467 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:44.665061 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:44.666379 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:44.668233 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:44.668908 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:44.669079 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:44.670363 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:44.670370 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:44.670384 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:44.670898 (+ 514us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:44.670995 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:44.671025 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:44.671139 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:44.671447 (+ 308us) server_negotiation.cc:299] Negotiation successful
0926 15:07:44.671590 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":367,"threads_started":1}
I20260926 15:07:44.675663 30312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3b70ed5442644138d8bb5141e04778b: No bootstrap required, opened a new log
I20260926 15:07:44.676627 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:07:44.685804 30312 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3b70ed5442644138d8bb5141e04778b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b70ed5442644138d8bb5141e04778b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42799 } }
I20260926 15:07:44.686362 30312 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3b70ed5442644138d8bb5141e04778b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:44.686512 30312 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3b70ed5442644138d8bb5141e04778b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3b70ed5442644138d8bb5141e04778b, State: Initialized, Role: FOLLOWER
I20260926 15:07:44.687132 30312 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3b70ed5442644138d8bb5141e04778b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b70ed5442644138d8bb5141e04778b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42799 } }
I20260926 15:07:44.687415 30312 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3b70ed5442644138d8bb5141e04778b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:07:44.687579 30312 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3b70ed5442644138d8bb5141e04778b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:07:44.687785 30312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3b70ed5442644138d8bb5141e04778b [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:44.688158 30315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:44.681767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59813 (local address 127.0.0.1:42799)
0926 15:07:44.681962 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:44.681972 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:44.682258 (+ 286us) server_negotiation.cc:407] Connection header received
0926 15:07:44.682422 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:44.682427 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:44.682505 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:44.682650 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:44.683097 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:44.684225 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:44.685552 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:44.686154 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:44.686288 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:44.687373 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:44.687379 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:44.687383 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:44.687410 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:44.687460 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:44.687482 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:44.687626 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:44.687788 (+ 162us) server_negotiation.cc:299] Negotiation successful
0926 15:07:44.687846 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 15:07:44.690469 30312 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3b70ed5442644138d8bb5141e04778b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b70ed5442644138d8bb5141e04778b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42799 } }
I20260926 15:07:44.690943 30312 leader_election.cc:302] T 00000000000000000000000000000000 P e3b70ed5442644138d8bb5141e04778b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3b70ed5442644138d8bb5141e04778b; no voters:
I20260926 15:07:44.691514 30312 leader_election.cc:288] T 00000000000000000000000000000000 P e3b70ed5442644138d8bb5141e04778b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:07:44.691663 30322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3b70ed5442644138d8bb5141e04778b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:07:44.692281 30322 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3b70ed5442644138d8bb5141e04778b [term 1 LEADER]: Becoming Leader. State: Replica: e3b70ed5442644138d8bb5141e04778b, State: Running, Role: LEADER
I20260926 15:07:44.692836 30322 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3b70ed5442644138d8bb5141e04778b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b70ed5442644138d8bb5141e04778b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42799 } }
I20260926 15:07:44.693540 30312 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3b70ed5442644138d8bb5141e04778b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:07:44.696221 30324 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3b70ed5442644138d8bb5141e04778b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3b70ed5442644138d8bb5141e04778b. Latest consensus state: current_term: 1 leader_uuid: "e3b70ed5442644138d8bb5141e04778b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b70ed5442644138d8bb5141e04778b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42799 } } }
I20260926 15:07:44.696581 30324 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3b70ed5442644138d8bb5141e04778b [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:44.697338 30323 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3b70ed5442644138d8bb5141e04778b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3b70ed5442644138d8bb5141e04778b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b70ed5442644138d8bb5141e04778b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42799 } } }
I20260926 15:07:44.697659 30323 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3b70ed5442644138d8bb5141e04778b [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:44.698599 30330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:07:44.703671 30330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:07:44.714339 30330 catalog_manager.cc:1347] Generated new cluster ID: 66355c0cf6c14a94af6e395ce09c4838
I20260926 15:07:44.714474 30330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:07:44.729600 30330 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:07:44.730948 30330 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:07:44.742357 30330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3b70ed5442644138d8bb5141e04778b: Generated new TSK 0
I20260926 15:07:44.743245 30330 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:07:44.750144 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 30245
W20260926 15:07:44.751509 30307 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33045
W20260926 15:07:44.752233 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 30245: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 15:07:44.752437 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 30245
W20260926 15:07:44.766193 30230 connection.cc:535] client connection to 127.0.0.1:42799 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42799): Connection reset by peer (error 104)
2026-09-26T15:07:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1817 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:07:44 dist-test-slave-7153 krb5kdc[30343](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:07:44 dist-test-slave-7153 krb5kdc[30343](info): set up 2 sockets
Sep 26 15:07:44 dist-test-slave-7153 krb5kdc[30343](info): commencing operation
krb5kdc: starting...
W20260926 15:07:45.777338 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.993s user 0.003s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:07:45 dist-test-slave-7153 krb5kdc[30343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:07:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:07:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 15:07:46.029218 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36269
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:07:46.365038 30356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:46.365391 30356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:46.365514 30356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:46.373515 30356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:07:46.373656 30356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:46.373703 30356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:46.373764 30356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:07:46.373806 30356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:07:46.384153 30356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36269
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:46.385512 30356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:46.386842 30356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:46.399024 30362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:46.399080 30361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:46.399120 30364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:46.399694 30356 server_base.cc:1034] running on GCE node
I20260926 15:07:46.400696 30356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:46.402408 30356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:46.403667 30356 hybrid_clock.cc:648] HybridClock initialized: now 1790435266403607 us; error 72 us; skew 500 ppm
I20260926 15:07:46.404105 30356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:46 dist-test-slave-7153 krb5kdc[30343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435266, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:46.415697 30356 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 15:07:46.417388 30356 webserver.cc:460] Webserver started at http://127.0.0.1:38241/ using document root <none> and password file <none>
I20260926 15:07:46.418114 30356 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:46.418224 30356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:46.418489 30356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:46.421685 30356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "8b348b296ab84dd58ddd1ac981272cf6"
format_stamp: "Formatted at 2026-09-26 15:07:46 on dist-test-slave-7153"
server_key: "43152216d0e507093f2b3ad706e81aad"
server_key_iv: "f57e5bd068fe3de104437c6a0ccb8b2b"
server_key_version: "encryptionkey@0"
I20260926 15:07:46.422458 30356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "8b348b296ab84dd58ddd1ac981272cf6"
format_stamp: "Formatted at 2026-09-26 15:07:46 on dist-test-slave-7153"
server_key: "43152216d0e507093f2b3ad706e81aad"
server_key_iv: "f57e5bd068fe3de104437c6a0ccb8b2b"
server_key_version: "encryptionkey@0"
I20260926 15:07:46.427970 30356 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 15:07:46.431660 30371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:46.433269 30356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260926 15:07:46.433475 30356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "8b348b296ab84dd58ddd1ac981272cf6"
format_stamp: "Formatted at 2026-09-26 15:07:46 on dist-test-slave-7153"
server_key: "43152216d0e507093f2b3ad706e81aad"
server_key_iv: "f57e5bd068fe3de104437c6a0ccb8b2b"
server_key_version: "encryptionkey@0"
I20260926 15:07:46.433682 30356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:46.439769 30356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:46.444087 30356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:46.444398 30356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:46.461864 30356 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36269
I20260926 15:07:46.461906 30422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36269 every 8 connection(s)
I20260926 15:07:46.463297 30356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:07:46.464059 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 30356
I20260926 15:07:46.464221 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:07:46.464715 9709 external_mini_cluster.cc:1428] Setting key 693f083cfacf2d23150110fd2cc23087
I20260926 15:07:46.467926 30423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:46.473012 30426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:46.466553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37355 (local address 127.0.0.1:36269)
0926 15:07:46.467401 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:46.467418 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:46.467472 (+ 54us) server_negotiation.cc:407] Connection header received
0926 15:07:46.469650 (+ 2178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:46.469673 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:46.470032 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:46.470370 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:46.471433 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:46.471439 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:46.471456 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:46.471470 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:46.471551 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:46.471573 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:46.471735 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:46.471843 (+ 108us) server_negotiation.cc:299] Negotiation successful
0926 15:07:46.472027 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":325,"threads_started":1}
I20260926 15:07:46.476789 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:07:46.481014 30423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b348b296ab84dd58ddd1ac981272cf6: Bootstrap starting.
I20260926 15:07:46.484692 30426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:46.482201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44897 (local address 127.0.0.1:36269)
0926 15:07:46.482399 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:46.482405 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:46.482766 (+ 361us) server_negotiation.cc:407] Connection header received
0926 15:07:46.482912 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:46.482917 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:46.482985 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:46.483100 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:46.484049 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:46.484055 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:46.484060 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:46.484078 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:46.484125 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:46.484145 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:46.484248 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:46.484350 (+ 102us) server_negotiation.cc:299] Negotiation successful
0926 15:07:46.484415 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 15:07:46.484877 30423 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b348b296ab84dd58ddd1ac981272cf6: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:46.486040 30423 log.cc:824] T 00000000000000000000000000000000 P 8b348b296ab84dd58ddd1ac981272cf6: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:46.488941 30423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b348b296ab84dd58ddd1ac981272cf6: No bootstrap required, opened a new log
I20260926 15:07:46.496398 30423 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b348b296ab84dd58ddd1ac981272cf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b348b296ab84dd58ddd1ac981272cf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36269 } }
I20260926 15:07:46.496838 30423 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b348b296ab84dd58ddd1ac981272cf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:46.497053 30423 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b348b296ab84dd58ddd1ac981272cf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b348b296ab84dd58ddd1ac981272cf6, State: Initialized, Role: FOLLOWER
I20260926 15:07:46.497824 30423 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b348b296ab84dd58ddd1ac981272cf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b348b296ab84dd58ddd1ac981272cf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36269 } }
I20260926 15:07:46.498103 30423 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b348b296ab84dd58ddd1ac981272cf6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:07:46.498245 30423 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b348b296ab84dd58ddd1ac981272cf6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:07:46.498412 30423 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b348b296ab84dd58ddd1ac981272cf6 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:46.500597 30423 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b348b296ab84dd58ddd1ac981272cf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b348b296ab84dd58ddd1ac981272cf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36269 } }
I20260926 15:07:46.501130 30423 leader_election.cc:302] T 00000000000000000000000000000000 P 8b348b296ab84dd58ddd1ac981272cf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b348b296ab84dd58ddd1ac981272cf6; no voters:
I20260926 15:07:46.501737 30423 leader_election.cc:288] T 00000000000000000000000000000000 P 8b348b296ab84dd58ddd1ac981272cf6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:07:46.501863 30433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b348b296ab84dd58ddd1ac981272cf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:07:46.502511 30433 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b348b296ab84dd58ddd1ac981272cf6 [term 1 LEADER]: Becoming Leader. State: Replica: 8b348b296ab84dd58ddd1ac981272cf6, State: Running, Role: LEADER
I20260926 15:07:46.503015 30433 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b348b296ab84dd58ddd1ac981272cf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b348b296ab84dd58ddd1ac981272cf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36269 } }
I20260926 15:07:46.503736 30423 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b348b296ab84dd58ddd1ac981272cf6 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:07:46.506891 30434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b348b296ab84dd58ddd1ac981272cf6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b348b296ab84dd58ddd1ac981272cf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b348b296ab84dd58ddd1ac981272cf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36269 } } }
I20260926 15:07:46.506649 30435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b348b296ab84dd58ddd1ac981272cf6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b348b296ab84dd58ddd1ac981272cf6. Latest consensus state: current_term: 1 leader_uuid: "8b348b296ab84dd58ddd1ac981272cf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b348b296ab84dd58ddd1ac981272cf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36269 } } }
I20260926 15:07:46.507194 30434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b348b296ab84dd58ddd1ac981272cf6 [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:46.507191 30435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b348b296ab84dd58ddd1ac981272cf6 [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:46.508836 30442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:07:46.513008 30442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:07:46.521683 30442 catalog_manager.cc:1347] Generated new cluster ID: 02686d1ca87849f98ab45599bddb02b9
I20260926 15:07:46.521808 30442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:07:46.537871 30442 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:07:46.538436 30442 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:07:46.548023 30442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b348b296ab84dd58ddd1ac981272cf6: Generated new TSK 0
I20260926 15:07:46.549336 30442 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:07:46.559474 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 30356
W20260926 15:07:46.560837 30418 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37355
W20260926 15:07:46.561779 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 30356: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 15:07:46.561990 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 30356
2026-09-26T15:07:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1810 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:07:46 dist-test-slave-7153 krb5kdc[30454](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:07:46 dist-test-slave-7153 krb5kdc[30454](info): set up 2 sockets
Sep 26 15:07:46 dist-test-slave-7153 krb5kdc[30454](info): commencing operation
krb5kdc: starting...
W20260926 15:07:47.593379 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.997s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:07:47 dist-test-slave-7153 krb5kdc[30454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:07:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:07:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 15:07:47.844887 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45531
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:07:48.177536 30467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:48.177838 30467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:48.177958 30467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:48.185217 30467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:07:48.185340 30467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:48.185398 30467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:48.185456 30467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:07:48.185498 30467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:07:48.195729 30467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:48.196992 30467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:48.198338 30467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:48.209630 30472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:48.209661 30475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:48.209726 30473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:48.210544 30467 server_base.cc:1034] running on GCE node
I20260926 15:07:48.211359 30467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:48.212920 30467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:48.214164 30467 hybrid_clock.cc:648] HybridClock initialized: now 1790435268214096 us; error 55 us; skew 500 ppm
I20260926 15:07:48.214588 30467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:48 dist-test-slave-7153 krb5kdc[30454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435268, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:48.218230 30467 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 15:07:48.219847 30467 webserver.cc:460] Webserver started at http://127.0.0.1:34309/ using document root <none> and password file <none>
I20260926 15:07:48.220587 30467 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:48.220713 30467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:48.221006 30467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:48.224174 30467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "b29cac4f8af144d7a51fbd4ddffcda0f"
format_stamp: "Formatted at 2026-09-26 15:07:48 on dist-test-slave-7153"
server_key: "e01b37b135b610f865a66b296c558b0b"
server_key_iv: "c2b2e058f1aad6ee149d8987ccd702bc"
server_key_version: "encryptionkey@0"
I20260926 15:07:48.225081 30467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "b29cac4f8af144d7a51fbd4ddffcda0f"
format_stamp: "Formatted at 2026-09-26 15:07:48 on dist-test-slave-7153"
server_key: "e01b37b135b610f865a66b296c558b0b"
server_key_iv: "c2b2e058f1aad6ee149d8987ccd702bc"
server_key_version: "encryptionkey@0"
I20260926 15:07:48.230747 30467 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260926 15:07:48.234604 30482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:48.236088 30467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:07:48.236286 30467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "b29cac4f8af144d7a51fbd4ddffcda0f"
format_stamp: "Formatted at 2026-09-26 15:07:48 on dist-test-slave-7153"
server_key: "e01b37b135b610f865a66b296c558b0b"
server_key_iv: "c2b2e058f1aad6ee149d8987ccd702bc"
server_key_version: "encryptionkey@0"
I20260926 15:07:48.236501 30467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:48.249748 30467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:48.254130 30467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:48.254441 30467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:48.272491 30467 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45531
I20260926 15:07:48.272541 30533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45531 every 8 connection(s)
I20260926 15:07:48.273918 30467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:07:48.277908 30534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:48.279512 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 30467
I20260926 15:07:48.279711 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:07:48.280256 9709 external_mini_cluster.cc:1428] Setting key ca311d9b1f9c3ad24f8c4103467fa121
I20260926 15:07:48.293300 30534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b29cac4f8af144d7a51fbd4ddffcda0f: Bootstrap starting.
Sep 26 15:07:48 dist-test-slave-7153 krb5kdc[30454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 15:07:48.297286 30534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b29cac4f8af144d7a51fbd4ddffcda0f: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:48.298527 30534 log.cc:824] T 00000000000000000000000000000000 P b29cac4f8af144d7a51fbd4ddffcda0f: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:48.302026 30534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b29cac4f8af144d7a51fbd4ddffcda0f: No bootstrap required, opened a new log
I20260926 15:07:48.304559 30537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:48.282111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38453 (local address 127.0.0.1:45531)
0926 15:07:48.282913 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:48.282925 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:48.282972 (+ 47us) server_negotiation.cc:407] Connection header received
0926 15:07:48.284564 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:48.284590 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:48.285041 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:48.285453 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:48.287205 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:48.288560 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:48.290663 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:48.291615 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:48.291764 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:48.296711 (+ 4947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:48.296721 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:48.296737 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:48.296779 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:48.300191 (+ 3412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:48.301171 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:48.301182 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:48.301189 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:48.301367 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:48.301826 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:48.301830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:48.301832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:48.302412 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:48.302645 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:48.303122 (+ 477us) server_negotiation.cc:299] Negotiation successful
0926 15:07:48.303481 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":358,"threads_started":1}
I20260926 15:07:48.308324 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:07:48.311160 30534 raft_consensus.cc:348] T 00000000000000000000000000000000 P b29cac4f8af144d7a51fbd4ddffcda0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b29cac4f8af144d7a51fbd4ddffcda0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45531 } }
I20260926 15:07:48.311676 30534 raft_consensus.cc:374] T 00000000000000000000000000000000 P b29cac4f8af144d7a51fbd4ddffcda0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:48.311784 30534 raft_consensus.cc:729] T 00000000000000000000000000000000 P b29cac4f8af144d7a51fbd4ddffcda0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b29cac4f8af144d7a51fbd4ddffcda0f, State: Initialized, Role: FOLLOWER
I20260926 15:07:48.312541 30534 consensus_queue.cc:260] T 00000000000000000000000000000000 P b29cac4f8af144d7a51fbd4ddffcda0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b29cac4f8af144d7a51fbd4ddffcda0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45531 } }
I20260926 15:07:48.312821 30534 raft_consensus.cc:388] T 00000000000000000000000000000000 P b29cac4f8af144d7a51fbd4ddffcda0f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:07:48.312978 30534 raft_consensus.cc:482] T 00000000000000000000000000000000 P b29cac4f8af144d7a51fbd4ddffcda0f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:07:48.313187 30534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b29cac4f8af144d7a51fbd4ddffcda0f [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:48.315799 30534 raft_consensus.cc:504] T 00000000000000000000000000000000 P b29cac4f8af144d7a51fbd4ddffcda0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b29cac4f8af144d7a51fbd4ddffcda0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45531 } }
I20260926 15:07:48.316382 30534 leader_election.cc:302] T 00000000000000000000000000000000 P b29cac4f8af144d7a51fbd4ddffcda0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b29cac4f8af144d7a51fbd4ddffcda0f; no voters:
I20260926 15:07:48.317018 30534 leader_election.cc:288] T 00000000000000000000000000000000 P b29cac4f8af144d7a51fbd4ddffcda0f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:07:48.317170 30544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b29cac4f8af144d7a51fbd4ddffcda0f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:07:48.318343 30544 raft_consensus.cc:686] T 00000000000000000000000000000000 P b29cac4f8af144d7a51fbd4ddffcda0f [term 1 LEADER]: Becoming Leader. State: Replica: b29cac4f8af144d7a51fbd4ddffcda0f, State: Running, Role: LEADER
I20260926 15:07:48.318990 30544 consensus_queue.cc:237] T 00000000000000000000000000000000 P b29cac4f8af144d7a51fbd4ddffcda0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b29cac4f8af144d7a51fbd4ddffcda0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45531 } }
I20260926 15:07:48.319341 30534 sys_catalog.cc:564] T 00000000000000000000000000000000 P b29cac4f8af144d7a51fbd4ddffcda0f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:07:48.322137 30545 sys_catalog.cc:455] T 00000000000000000000000000000000 P b29cac4f8af144d7a51fbd4ddffcda0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b29cac4f8af144d7a51fbd4ddffcda0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b29cac4f8af144d7a51fbd4ddffcda0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45531 } } }
I20260926 15:07:48.322546 30545 sys_catalog.cc:458] T 00000000000000000000000000000000 P b29cac4f8af144d7a51fbd4ddffcda0f [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:48.328029 30546 sys_catalog.cc:455] T 00000000000000000000000000000000 P b29cac4f8af144d7a51fbd4ddffcda0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b29cac4f8af144d7a51fbd4ddffcda0f. Latest consensus state: current_term: 1 leader_uuid: "b29cac4f8af144d7a51fbd4ddffcda0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b29cac4f8af144d7a51fbd4ddffcda0f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45531 } } }
I20260926 15:07:48.328980 30546 sys_catalog.cc:458] T 00000000000000000000000000000000 P b29cac4f8af144d7a51fbd4ddffcda0f [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:48.329936 30550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:07:48.330837 30537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:48.313727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51079 (local address 127.0.0.1:45531)
0926 15:07:48.313934 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:48.313941 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:48.314325 (+ 384us) server_negotiation.cc:407] Connection header received
0926 15:07:48.314481 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:48.314487 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:48.314572 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:48.314690 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:48.316200 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:48.317312 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:48.319163 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:48.319796 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:48.319945 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:48.323146 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:48.323154 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:48.323160 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:48.323211 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:48.327294 (+ 4083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:48.328365 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:48.328371 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:48.328375 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:48.328579 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:48.329154 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:48.329161 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:48.329165 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:48.329673 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:48.329915 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:48.330258 (+ 343us) server_negotiation.cc:299] Negotiation successful
0926 15:07:48.330484 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 15:07:48.336185 30550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:07:48.347177 30550 catalog_manager.cc:1347] Generated new cluster ID: b86ac649564b49fe8c4c7887ac76b867
I20260926 15:07:48.347326 30550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:07:48.382588 30550 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:07:48.384526 30550 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:07:48.404572 30550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b29cac4f8af144d7a51fbd4ddffcda0f: Generated new TSK 0
I20260926 15:07:48.405680 30550 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:07:48.422470 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 30467
I20260926 15:07:48.423676 30529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:48.521238 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 30467
2026-09-26T15:07:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1958 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:07:48 dist-test-slave-7153 krb5kdc[30566](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:07:48 dist-test-slave-7153 krb5kdc[30566](info): set up 2 sockets
Sep 26 15:07:48 dist-test-slave-7153 krb5kdc[30566](info): commencing operation
krb5kdc: starting...
W20260926 15:07:49.549474 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.995s user 0.006s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:07:49 dist-test-slave-7153 krb5kdc[30566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:07:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:07:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 15:07:49.808437 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43013
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:07:50.140027 30579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:50.140422 30579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:50.140566 30579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:50.147913 30579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:07:50.148061 30579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:50.148138 30579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:50.148227 30579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:07:50.148290 30579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:07:50.158460 30579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:50.159792 30579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:50.161158 30579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:50.173942 30584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:50.173926 30585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:50.173995 30587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:50.174716 30579 server_base.cc:1034] running on GCE node
I20260926 15:07:50.175583 30579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:50.177085 30579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:50.178344 30579 hybrid_clock.cc:648] HybridClock initialized: now 1790435270178289 us; error 45 us; skew 500 ppm
I20260926 15:07:50.178699 30579 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:50 dist-test-slave-7153 krb5kdc[30566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435270, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:50.181818 30579 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 15:07:50.183339 30579 webserver.cc:460] Webserver started at http://127.0.0.1:37245/ using document root <none> and password file <none>
I20260926 15:07:50.184023 30579 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:50.184136 30579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:50.184428 30579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:50.187475 30579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "6ea3bd575d6443b0ac2dfcc549c8b844"
format_stamp: "Formatted at 2026-09-26 15:07:50 on dist-test-slave-7153"
server_key: "99a88108f88184c011e4fa7b7829e8eb"
server_key_iv: "c5c5a1bb5b78b55540b72c31ee4b6324"
server_key_version: "encryptionkey@0"
I20260926 15:07:50.188226 30579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "6ea3bd575d6443b0ac2dfcc549c8b844"
format_stamp: "Formatted at 2026-09-26 15:07:50 on dist-test-slave-7153"
server_key: "99a88108f88184c011e4fa7b7829e8eb"
server_key_iv: "c5c5a1bb5b78b55540b72c31ee4b6324"
server_key_version: "encryptionkey@0"
I20260926 15:07:50.193769 30579 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260926 15:07:50.197363 30594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:50.198773 30579 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:07:50.198945 30579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "6ea3bd575d6443b0ac2dfcc549c8b844"
format_stamp: "Formatted at 2026-09-26 15:07:50 on dist-test-slave-7153"
server_key: "99a88108f88184c011e4fa7b7829e8eb"
server_key_iv: "c5c5a1bb5b78b55540b72c31ee4b6324"
server_key_version: "encryptionkey@0"
I20260926 15:07:50.199132 30579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:50.231881 30579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:50.236783 30579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:50.237092 30579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:50.255445 30579 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43013
I20260926 15:07:50.255482 30645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43013 every 8 connection(s)
I20260926 15:07:50.256803 30579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:07:50.260628 30646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:50.262614 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 30579
I20260926 15:07:50.262816 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:07:50.263231 9709 external_mini_cluster.cc:1428] Setting key b382ab22d2abaeea3bced0515203c2c1
I20260926 15:07:50.275444 30646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ea3bd575d6443b0ac2dfcc549c8b844: Bootstrap starting.
I20260926 15:07:50.278676 30649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:50.265155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39959 (local address 127.0.0.1:43013)
0926 15:07:50.266038 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:50.266054 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:50.266113 (+ 59us) server_negotiation.cc:407] Connection header received
0926 15:07:50.267904 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:50.267953 (+ 49us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:50.268582 (+ 629us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:50.269059 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:50.269631 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:50.271504 (+ 1873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:50.273129 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:50.274254 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:50.274458 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:50.275659 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:50.275669 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:50.275688 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:50.276419 (+ 731us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:50.276525 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:50.276559 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:50.276692 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:50.276930 (+ 238us) server_negotiation.cc:299] Negotiation successful
0926 15:07:50.277300 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":372,"threads_started":1}
I20260926 15:07:50.279547 30646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ea3bd575d6443b0ac2dfcc549c8b844: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:50.280666 30646 log.cc:824] T 00000000000000000000000000000000 P 6ea3bd575d6443b0ac2dfcc549c8b844: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:50.282644 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:07:50.283779 30646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ea3bd575d6443b0ac2dfcc549c8b844: No bootstrap required, opened a new log
I20260926 15:07:50.292039 30646 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ea3bd575d6443b0ac2dfcc549c8b844 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ea3bd575d6443b0ac2dfcc549c8b844" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43013 } }
I20260926 15:07:50.292507 30646 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ea3bd575d6443b0ac2dfcc549c8b844 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:50.292651 30646 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ea3bd575d6443b0ac2dfcc549c8b844 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ea3bd575d6443b0ac2dfcc549c8b844, State: Initialized, Role: FOLLOWER
I20260926 15:07:50.293144 30649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:50.286956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38647 (local address 127.0.0.1:43013)
0926 15:07:50.287114 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:50.287120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:50.287589 (+ 469us) server_negotiation.cc:407] Connection header received
0926 15:07:50.287795 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:50.287799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:50.287870 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:50.287956 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:50.288498 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:50.289322 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:50.290720 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:50.291198 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:50.291310 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:50.292423 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:50.292428 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:50.292431 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:50.292450 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:50.292499 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:50.292518 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:50.292668 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:50.292719 (+ 51us) server_negotiation.cc:299] Negotiation successful
0926 15:07:50.292814 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260926 15:07:50.293474 30646 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ea3bd575d6443b0ac2dfcc549c8b844 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ea3bd575d6443b0ac2dfcc549c8b844" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43013 } }
I20260926 15:07:50.293723 30646 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ea3bd575d6443b0ac2dfcc549c8b844 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:07:50.293869 30646 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ea3bd575d6443b0ac2dfcc549c8b844 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:07:50.294028 30646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ea3bd575d6443b0ac2dfcc549c8b844 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:50.296370 30646 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ea3bd575d6443b0ac2dfcc549c8b844 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ea3bd575d6443b0ac2dfcc549c8b844" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43013 } }
I20260926 15:07:50.296890 30646 leader_election.cc:302] T 00000000000000000000000000000000 P 6ea3bd575d6443b0ac2dfcc549c8b844 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ea3bd575d6443b0ac2dfcc549c8b844; no voters:
I20260926 15:07:50.297529 30646 leader_election.cc:288] T 00000000000000000000000000000000 P 6ea3bd575d6443b0ac2dfcc549c8b844 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:07:50.297650 30656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ea3bd575d6443b0ac2dfcc549c8b844 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:07:50.299700 30646 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ea3bd575d6443b0ac2dfcc549c8b844 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:07:50.304450 30656 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ea3bd575d6443b0ac2dfcc549c8b844 [term 1 LEADER]: Becoming Leader. State: Replica: 6ea3bd575d6443b0ac2dfcc549c8b844, State: Running, Role: LEADER
I20260926 15:07:50.304987 30656 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ea3bd575d6443b0ac2dfcc549c8b844 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ea3bd575d6443b0ac2dfcc549c8b844" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43013 } }
I20260926 15:07:50.307993 30658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ea3bd575d6443b0ac2dfcc549c8b844 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ea3bd575d6443b0ac2dfcc549c8b844. Latest consensus state: current_term: 1 leader_uuid: "6ea3bd575d6443b0ac2dfcc549c8b844" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ea3bd575d6443b0ac2dfcc549c8b844" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43013 } } }
I20260926 15:07:50.308468 30658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ea3bd575d6443b0ac2dfcc549c8b844 [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:50.308032 30657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ea3bd575d6443b0ac2dfcc549c8b844 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ea3bd575d6443b0ac2dfcc549c8b844" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ea3bd575d6443b0ac2dfcc549c8b844" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43013 } } }
I20260926 15:07:50.308897 30657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ea3bd575d6443b0ac2dfcc549c8b844 [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:50.310664 30671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:07:50.315936 30671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:07:50.324885 30671 catalog_manager.cc:1347] Generated new cluster ID: 8324a173cdb2420f97236f880cda3557
I20260926 15:07:50.325016 30671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:07:50.353418 30671 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:07:50.355319 30671 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:07:50.371659 30671 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ea3bd575d6443b0ac2dfcc549c8b844: Generated new TSK 0
I20260926 15:07:50.372838 30671 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:07:50.388499 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 30579
W20260926 15:07:50.390224 30641 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39959
W20260926 15:07:50.390983 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 30579: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 15:07:50.391127 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 30579
2026-09-26T15:07:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1869 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:07:50 dist-test-slave-7153 krb5kdc[30677](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:07:50 dist-test-slave-7153 krb5kdc[30677](info): set up 2 sockets
Sep 26 15:07:50 dist-test-slave-7153 krb5kdc[30677](info): commencing operation
krb5kdc: starting...
W20260926 15:07:51.413385 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.990s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:07:51 dist-test-slave-7153 krb5kdc[30677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:07:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:07:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 15:07:51.657560 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34673
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:07:51.985430 30690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:51.985803 30690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:51.985953 30690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:51.993235 30690 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:07:51.993363 30690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:51.993441 30690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:51.993521 30690 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:07:51.993590 30690 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:07:52.003446 30690 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:52.004731 30690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:52.006040 30690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:52.017369 30695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:52.017308 30698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:52.017338 30696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:52.017721 30690 server_base.cc:1034] running on GCE node
I20260926 15:07:52.018814 30690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:52.020243 30690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:52.021494 30690 hybrid_clock.cc:648] HybridClock initialized: now 1790435272021430 us; error 50 us; skew 500 ppm
I20260926 15:07:52.021845 30690 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:52 dist-test-slave-7153 krb5kdc[30677](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435272, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:52.030503 30690 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 15:07:52.032104 30690 webserver.cc:460] Webserver started at http://127.0.0.1:42417/ using document root <none> and password file <none>
I20260926 15:07:52.032809 30690 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:52.032928 30690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:52.033273 30690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:52.036316 30690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "d206a50761a148348a78866df08ff767"
format_stamp: "Formatted at 2026-09-26 15:07:52 on dist-test-slave-7153"
server_key: "0c13ee6557dab4f5da53718265a50523"
server_key_iv: "8df87725a90f56e36ff27d72b7a95c91"
server_key_version: "encryptionkey@0"
I20260926 15:07:52.037088 30690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "d206a50761a148348a78866df08ff767"
format_stamp: "Formatted at 2026-09-26 15:07:52 on dist-test-slave-7153"
server_key: "0c13ee6557dab4f5da53718265a50523"
server_key_iv: "8df87725a90f56e36ff27d72b7a95c91"
server_key_version: "encryptionkey@0"
I20260926 15:07:52.042488 30690 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 15:07:52.046178 30705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:52.047688 30690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:07:52.047869 30690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "d206a50761a148348a78866df08ff767"
format_stamp: "Formatted at 2026-09-26 15:07:52 on dist-test-slave-7153"
server_key: "0c13ee6557dab4f5da53718265a50523"
server_key_iv: "8df87725a90f56e36ff27d72b7a95c91"
server_key_version: "encryptionkey@0"
I20260926 15:07:52.048074 30690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:52.054162 30690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:52.058107 30690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:52.058383 30690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:52.076136 30690 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34673
I20260926 15:07:52.076180 30756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34673 every 8 connection(s)
I20260926 15:07:52.077729 30690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:07:52.080787 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 30690
I20260926 15:07:52.080972 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:07:52.081535 9709 external_mini_cluster.cc:1428] Setting key 2639c44f7df09edff0795ba84f8f2f09
I20260926 15:07:52.082136 30757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:52.089155 30760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:52.083347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36703 (local address 127.0.0.1:34673)
0926 15:07:52.084153 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:52.084166 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:52.084207 (+ 41us) server_negotiation.cc:407] Connection header received
0926 15:07:52.085775 (+ 1568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:52.085797 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:52.086169 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:52.086505 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:52.087564 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:52.087570 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:52.087584 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:52.087598 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:52.087666 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:52.087688 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:52.087857 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:52.087952 (+ 95us) server_negotiation.cc:299] Negotiation successful
0926 15:07:52.088141 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":359,"threads_started":1}
I20260926 15:07:52.092954 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:07:52.096359 30757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d206a50761a148348a78866df08ff767: Bootstrap starting.
I20260926 15:07:52.100350 30757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d206a50761a148348a78866df08ff767: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:52.100392 30760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:52.098049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48841 (local address 127.0.0.1:34673)
0926 15:07:52.098207 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:52.098211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:52.098564 (+ 353us) server_negotiation.cc:407] Connection header received
0926 15:07:52.098739 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:52.098742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:52.098791 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:52.098871 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:52.099792 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:52.099796 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:52.099799 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:52.099814 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:52.099854 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:52.099868 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:52.099951 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:52.100080 (+ 129us) server_negotiation.cc:299] Negotiation successful
0926 15:07:52.100124 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260926 15:07:52.101522 30757 log.cc:824] T 00000000000000000000000000000000 P d206a50761a148348a78866df08ff767: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:52.104176 30757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d206a50761a148348a78866df08ff767: No bootstrap required, opened a new log
I20260926 15:07:52.111379 30757 raft_consensus.cc:348] T 00000000000000000000000000000000 P d206a50761a148348a78866df08ff767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d206a50761a148348a78866df08ff767" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34673 } }
I20260926 15:07:52.111783 30757 raft_consensus.cc:374] T 00000000000000000000000000000000 P d206a50761a148348a78866df08ff767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:52.111877 30757 raft_consensus.cc:729] T 00000000000000000000000000000000 P d206a50761a148348a78866df08ff767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d206a50761a148348a78866df08ff767, State: Initialized, Role: FOLLOWER
I20260926 15:07:52.112448 30757 consensus_queue.cc:260] T 00000000000000000000000000000000 P d206a50761a148348a78866df08ff767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d206a50761a148348a78866df08ff767" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34673 } }
I20260926 15:07:52.112680 30757 raft_consensus.cc:388] T 00000000000000000000000000000000 P d206a50761a148348a78866df08ff767 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:07:52.112805 30757 raft_consensus.cc:482] T 00000000000000000000000000000000 P d206a50761a148348a78866df08ff767 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:07:52.112967 30757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d206a50761a148348a78866df08ff767 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:52.115147 30757 raft_consensus.cc:504] T 00000000000000000000000000000000 P d206a50761a148348a78866df08ff767 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d206a50761a148348a78866df08ff767" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34673 } }
I20260926 15:07:52.115633 30757 leader_election.cc:302] T 00000000000000000000000000000000 P d206a50761a148348a78866df08ff767 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d206a50761a148348a78866df08ff767; no voters:
I20260926 15:07:52.116185 30757 leader_election.cc:288] T 00000000000000000000000000000000 P d206a50761a148348a78866df08ff767 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:07:52.116309 30767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d206a50761a148348a78866df08ff767 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:07:52.116918 30767 raft_consensus.cc:686] T 00000000000000000000000000000000 P d206a50761a148348a78866df08ff767 [term 1 LEADER]: Becoming Leader. State: Replica: d206a50761a148348a78866df08ff767, State: Running, Role: LEADER
I20260926 15:07:52.117444 30767 consensus_queue.cc:237] T 00000000000000000000000000000000 P d206a50761a148348a78866df08ff767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d206a50761a148348a78866df08ff767" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34673 } }
I20260926 15:07:52.118175 30757 sys_catalog.cc:564] T 00000000000000000000000000000000 P d206a50761a148348a78866df08ff767 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:07:52.120042 30769 sys_catalog.cc:455] T 00000000000000000000000000000000 P d206a50761a148348a78866df08ff767 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d206a50761a148348a78866df08ff767. Latest consensus state: current_term: 1 leader_uuid: "d206a50761a148348a78866df08ff767" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d206a50761a148348a78866df08ff767" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34673 } } }
I20260926 15:07:52.120486 30769 sys_catalog.cc:458] T 00000000000000000000000000000000 P d206a50761a148348a78866df08ff767 [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:52.120399 30768 sys_catalog.cc:455] T 00000000000000000000000000000000 P d206a50761a148348a78866df08ff767 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d206a50761a148348a78866df08ff767" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d206a50761a148348a78866df08ff767" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34673 } } }
I20260926 15:07:52.120702 30768 sys_catalog.cc:458] T 00000000000000000000000000000000 P d206a50761a148348a78866df08ff767 [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:52.121436 30775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:07:52.126362 30775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:07:52.134429 30775 catalog_manager.cc:1347] Generated new cluster ID: 9f71c8f771464542acde42d47c27a5c1
I20260926 15:07:52.134546 30775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:07:52.160491 30775 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:07:52.161132 30775 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:07:52.176147 30775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d206a50761a148348a78866df08ff767: Generated new TSK 0
I20260926 15:07:52.177320 30775 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:07:52.195109 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 30690
W20260926 15:07:52.196432 30752 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36703
W20260926 15:07:52.197166 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 30690: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 15:07:52.197304 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 30690
2026-09-26T15:07:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1806 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:07:52 dist-test-slave-7153 krb5kdc[30788](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:07:52 dist-test-slave-7153 krb5kdc[30788](info): set up 2 sockets
Sep 26 15:07:52 dist-test-slave-7153 krb5kdc[30788](info): commencing operation
krb5kdc: starting...
W20260926 15:07:53.217360 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.987s user 0.005s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:07:53 dist-test-slave-7153 krb5kdc[30788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:07:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:07:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 15:07:53.467832 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43167
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:07:53.797317 30801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:53.797649 30801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:53.797784 30801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:53.805269 30801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:07:53.805418 30801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:53.805500 30801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:53.805583 30801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:07:53.805644 30801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:07:53.815660 30801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:53.816975 30801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:53.818317 30801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:53.830076 30806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:53.830169 30809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:53.830317 30807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:53.831003 30801 server_base.cc:1034] running on GCE node
I20260926 15:07:53.831799 30801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:53.833349 30801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:53.834590 30801 hybrid_clock.cc:648] HybridClock initialized: now 1790435273834533 us; error 61 us; skew 500 ppm
I20260926 15:07:53.834988 30801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:53 dist-test-slave-7153 krb5kdc[30788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435273, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:53.838083 30801 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 15:07:53.839654 30801 webserver.cc:460] Webserver started at http://127.0.0.1:34209/ using document root <none> and password file <none>
I20260926 15:07:53.840375 30801 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:53.840600 30801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:53.840914 30801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:53.844049 30801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "c3b7d4ee581d4bb1ad1a38c5a2604677"
format_stamp: "Formatted at 2026-09-26 15:07:53 on dist-test-slave-7153"
server_key: "e9f062045d2ede0a0d887f8270a16331"
server_key_iv: "1fa2b6ec84d151626556791742f58065"
server_key_version: "encryptionkey@0"
I20260926 15:07:53.844801 30801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "c3b7d4ee581d4bb1ad1a38c5a2604677"
format_stamp: "Formatted at 2026-09-26 15:07:53 on dist-test-slave-7153"
server_key: "e9f062045d2ede0a0d887f8270a16331"
server_key_iv: "1fa2b6ec84d151626556791742f58065"
server_key_version: "encryptionkey@0"
I20260926 15:07:53.850301 30801 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260926 15:07:53.854346 30816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:53.855886 30801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260926 15:07:53.856091 30801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "c3b7d4ee581d4bb1ad1a38c5a2604677"
format_stamp: "Formatted at 2026-09-26 15:07:53 on dist-test-slave-7153"
server_key: "e9f062045d2ede0a0d887f8270a16331"
server_key_iv: "1fa2b6ec84d151626556791742f58065"
server_key_version: "encryptionkey@0"
I20260926 15:07:53.856292 30801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:53.876168 30801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:53.880926 30801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:53.881279 30801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:53.899192 30801 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43167
I20260926 15:07:53.899228 30867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43167 every 8 connection(s)
I20260926 15:07:53.900633 30801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:07:53.901937 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 30801
I20260926 15:07:53.902096 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:07:53.902527 9709 external_mini_cluster.cc:1428] Setting key c3da482e7704f42027a255a85a8b491b
I20260926 15:07:53.905122 30868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 26 15:07:53 dist-test-slave-7153 krb5kdc[30788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 15:07:53.918850 30868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b7d4ee581d4bb1ad1a38c5a2604677: Bootstrap starting.
I20260926 15:07:53.923349 30868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3b7d4ee581d4bb1ad1a38c5a2604677: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:53.924672 30868 log.cc:824] T 00000000000000000000000000000000 P c3b7d4ee581d4bb1ad1a38c5a2604677: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:53.928694 30868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b7d4ee581d4bb1ad1a38c5a2604677: No bootstrap required, opened a new log
I20260926 15:07:53.929522 30871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:53.904346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.17:44291 (local address 127.0.0.1:43167)
0926 15:07:53.905152 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:53.905168 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:53.905225 (+ 57us) server_negotiation.cc:407] Connection header received
0926 15:07:53.907539 (+ 2314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:53.907573 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:53.908135 (+ 562us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:53.908661 (+ 526us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:53.910152 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:53.912233 (+ 2081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:53.913690 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:53.914736 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:53.914922 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:53.919234 (+ 4312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:53.919248 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:53.919272 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:53.919326 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:53.923630 (+ 4304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:53.924782 (+ 1152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:53.924792 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:53.924801 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:53.925033 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:53.925758 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:53.925767 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:53.925770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:53.926486 (+ 716us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:53.926857 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:53.927541 (+ 684us) server_negotiation.cc:299] Negotiation successful
0926 15:07:53.927956 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":312,"threads_started":1}
I20260926 15:07:53.934747 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:07:53.938405 30868 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3b7d4ee581d4bb1ad1a38c5a2604677 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b7d4ee581d4bb1ad1a38c5a2604677" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43167 } }
I20260926 15:07:53.938848 30868 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3b7d4ee581d4bb1ad1a38c5a2604677 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:53.938979 30868 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3b7d4ee581d4bb1ad1a38c5a2604677 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b7d4ee581d4bb1ad1a38c5a2604677, State: Initialized, Role: FOLLOWER
I20260926 15:07:53.939796 30868 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3b7d4ee581d4bb1ad1a38c5a2604677 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b7d4ee581d4bb1ad1a38c5a2604677" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43167 } }
I20260926 15:07:53.940160 30868 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3b7d4ee581d4bb1ad1a38c5a2604677 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:07:53.940326 30868 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3b7d4ee581d4bb1ad1a38c5a2604677 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:07:53.940534 30868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3b7d4ee581d4bb1ad1a38c5a2604677 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:53.943449 30868 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3b7d4ee581d4bb1ad1a38c5a2604677 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b7d4ee581d4bb1ad1a38c5a2604677" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43167 } }
I20260926 15:07:53.944133 30868 leader_election.cc:302] T 00000000000000000000000000000000 P c3b7d4ee581d4bb1ad1a38c5a2604677 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3b7d4ee581d4bb1ad1a38c5a2604677; no voters:
I20260926 15:07:53.944813 30868 leader_election.cc:288] T 00000000000000000000000000000000 P c3b7d4ee581d4bb1ad1a38c5a2604677 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:07:53.944937 30878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3b7d4ee581d4bb1ad1a38c5a2604677 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:07:53.946774 30878 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3b7d4ee581d4bb1ad1a38c5a2604677 [term 1 LEADER]: Becoming Leader. State: Replica: c3b7d4ee581d4bb1ad1a38c5a2604677, State: Running, Role: LEADER
I20260926 15:07:53.946954 30868 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3b7d4ee581d4bb1ad1a38c5a2604677 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:07:53.947404 30878 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3b7d4ee581d4bb1ad1a38c5a2604677 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b7d4ee581d4bb1ad1a38c5a2604677" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43167 } }
I20260926 15:07:53.949757 30880 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b7d4ee581d4bb1ad1a38c5a2604677 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3b7d4ee581d4bb1ad1a38c5a2604677. Latest consensus state: current_term: 1 leader_uuid: "c3b7d4ee581d4bb1ad1a38c5a2604677" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b7d4ee581d4bb1ad1a38c5a2604677" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43167 } } }
I20260926 15:07:53.950158 30880 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b7d4ee581d4bb1ad1a38c5a2604677 [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:53.950505 30879 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b7d4ee581d4bb1ad1a38c5a2604677 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3b7d4ee581d4bb1ad1a38c5a2604677" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b7d4ee581d4bb1ad1a38c5a2604677" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43167 } } }
I20260926 15:07:53.950798 30879 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b7d4ee581d4bb1ad1a38c5a2604677 [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:53.957135 30885 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:07:53.958249 30871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:53.939229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.17:40855 (local address 127.0.0.1:43167)
0926 15:07:53.939469 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:53.939477 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:53.939997 (+ 520us) server_negotiation.cc:407] Connection header received
0926 15:07:53.940206 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:53.940210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:53.940280 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:53.940380 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:53.942212 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:53.943059 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:53.944763 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:53.945531 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:53.945671 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:53.949165 (+ 3494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:53.949173 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:53.949179 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:53.949228 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:53.953600 (+ 4372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:53.954862 (+ 1262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:53.954867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:53.954870 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:53.955051 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:53.955717 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:53.955723 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:53.955727 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:53.956142 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:53.957438 (+ 1296us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:53.957616 (+ 178us) server_negotiation.cc:299] Negotiation successful
0926 15:07:53.957807 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 15:07:53.962500 30885 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:07:53.975920 30885 catalog_manager.cc:1347] Generated new cluster ID: d1745ed2a49544f1ae583237c9231294
I20260926 15:07:53.976051 30885 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:07:53.992237 30885 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:07:53.993873 30885 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:07:54.003880 30885 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3b7d4ee581d4bb1ad1a38c5a2604677: Generated new TSK 0
I20260926 15:07:54.004739 30885 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:07:54.017788 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 30801
I20260926 15:07:54.019011 30863 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:54.109858 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 30801
W20260926 15:07:54.123724 30786 connection.cc:535] client connection to 127.0.0.1:43167 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43167): Connection reset by peer (error 104)
2026-09-26T15:07:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1912 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:07:54 dist-test-slave-7153 krb5kdc[30900](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:07:54 dist-test-slave-7153 krb5kdc[30900](info): set up 2 sockets
Sep 26 15:07:54 dist-test-slave-7153 krb5kdc[30900](info): commencing operation
krb5kdc: starting...
W20260926 15:07:55.125516 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.983s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:07:55 dist-test-slave-7153 krb5kdc[30900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:07:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:07:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 15:07:55.381027 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39947
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:07:55.714690 30913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:55.714985 30913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:55.715123 30913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:55.722316 30913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:07:55.722437 30913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:55.722514 30913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:55.722599 30913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:07:55.722659 30913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:07:55.732631 30913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39947
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:55.734010 30913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:55.735375 30913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:55.747403 30921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:55.747366 30919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:55.747449 30918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:55.748327 30913 server_base.cc:1034] running on GCE node
I20260926 15:07:55.749169 30913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:55.750604 30913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:55.751816 30913 hybrid_clock.cc:648] HybridClock initialized: now 1790435275751764 us; error 43 us; skew 500 ppm
I20260926 15:07:55.752179 30913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:55 dist-test-slave-7153 krb5kdc[30900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435275, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:55.755436 30913 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 15:07:55.757017 30913 webserver.cc:460] Webserver started at http://127.0.0.1:43669/ using document root <none> and password file <none>
I20260926 15:07:55.757733 30913 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:55.757848 30913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:55.758129 30913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:55.761083 30913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "f5ee02b1aca346369fda5a09e646df0e"
format_stamp: "Formatted at 2026-09-26 15:07:55 on dist-test-slave-7153"
server_key: "f2ecdcbcea4ab16786eac90f89dd46c0"
server_key_iv: "0e3b2177e83c3f9ad096c7ec912c3d2c"
server_key_version: "encryptionkey@0"
I20260926 15:07:55.761912 30913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "f5ee02b1aca346369fda5a09e646df0e"
format_stamp: "Formatted at 2026-09-26 15:07:55 on dist-test-slave-7153"
server_key: "f2ecdcbcea4ab16786eac90f89dd46c0"
server_key_iv: "0e3b2177e83c3f9ad096c7ec912c3d2c"
server_key_version: "encryptionkey@0"
I20260926 15:07:55.767270 30913 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 15:07:55.770998 30928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:55.772361 30913 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 15:07:55.772542 30913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "f5ee02b1aca346369fda5a09e646df0e"
format_stamp: "Formatted at 2026-09-26 15:07:55 on dist-test-slave-7153"
server_key: "f2ecdcbcea4ab16786eac90f89dd46c0"
server_key_iv: "0e3b2177e83c3f9ad096c7ec912c3d2c"
server_key_version: "encryptionkey@0"
I20260926 15:07:55.772727 30913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:55.791832 30913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:55.796041 30913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:55.796304 30913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:55.813504 30913 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39947
I20260926 15:07:55.813525 30979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39947 every 8 connection(s)
I20260926 15:07:55.815002 30913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:07:55.815305 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 30913
I20260926 15:07:55.815495 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:07:55.816094 9709 external_mini_cluster.cc:1428] Setting key d8c6f696c0609b4dacc0e325a3f76cea
I20260926 15:07:55.820256 30980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:55.832672 30983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:55.818086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.17:34821 (local address 127.0.0.1:39947)
0926 15:07:55.819013 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:55.819030 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:55.819083 (+ 53us) server_negotiation.cc:407] Connection header received
0926 15:07:55.821436 (+ 2353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:55.821470 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:55.822072 (+ 602us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:55.822585 (+ 513us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:55.823140 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:55.825235 (+ 2095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:55.826948 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:55.828072 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:55.828299 (+ 227us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:55.829644 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:55.829655 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:55.829676 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:55.830440 (+ 764us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:55.830541 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:55.830581 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:55.830758 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:55.831030 (+ 272us) server_negotiation.cc:299] Negotiation successful
0926 15:07:55.831237 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":409,"threads_started":1}
I20260926 15:07:55.834832 30980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5ee02b1aca346369fda5a09e646df0e: Bootstrap starting.
I20260926 15:07:55.836889 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:07:55.838238 30980 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5ee02b1aca346369fda5a09e646df0e: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:55.839247 30980 log.cc:824] T 00000000000000000000000000000000 P f5ee02b1aca346369fda5a09e646df0e: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:55.842377 30980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5ee02b1aca346369fda5a09e646df0e: No bootstrap required, opened a new log
I20260926 15:07:55.847714 30983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:55.841150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.17:48881 (local address 127.0.0.1:39947)
0926 15:07:55.841323 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:55.841328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:55.841733 (+ 405us) server_negotiation.cc:407] Connection header received
0926 15:07:55.841894 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:55.841899 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:55.842005 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:55.842112 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:55.842577 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:55.843674 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:55.845008 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:55.845722 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:55.845886 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:55.846862 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:55.846869 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:55.846874 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:55.846899 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:55.846964 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:55.846989 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:55.847102 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:55.847316 (+ 214us) server_negotiation.cc:299] Negotiation successful
0926 15:07:55.847373 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 15:07:55.851315 30980 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5ee02b1aca346369fda5a09e646df0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ee02b1aca346369fda5a09e646df0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39947 } }
I20260926 15:07:55.851686 30980 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5ee02b1aca346369fda5a09e646df0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:55.851796 30980 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5ee02b1aca346369fda5a09e646df0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ee02b1aca346369fda5a09e646df0e, State: Initialized, Role: FOLLOWER
I20260926 15:07:55.852494 30980 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5ee02b1aca346369fda5a09e646df0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ee02b1aca346369fda5a09e646df0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39947 } }
I20260926 15:07:55.852703 30980 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5ee02b1aca346369fda5a09e646df0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:07:55.852801 30980 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5ee02b1aca346369fda5a09e646df0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:07:55.852921 30980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5ee02b1aca346369fda5a09e646df0e [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:55.855096 30980 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5ee02b1aca346369fda5a09e646df0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ee02b1aca346369fda5a09e646df0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39947 } }
I20260926 15:07:55.855677 30980 leader_election.cc:302] T 00000000000000000000000000000000 P f5ee02b1aca346369fda5a09e646df0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5ee02b1aca346369fda5a09e646df0e; no voters:
I20260926 15:07:55.856225 30980 leader_election.cc:288] T 00000000000000000000000000000000 P f5ee02b1aca346369fda5a09e646df0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:07:55.856348 30990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5ee02b1aca346369fda5a09e646df0e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:07:55.856997 30990 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5ee02b1aca346369fda5a09e646df0e [term 1 LEADER]: Becoming Leader. State: Replica: f5ee02b1aca346369fda5a09e646df0e, State: Running, Role: LEADER
I20260926 15:07:55.857597 30990 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5ee02b1aca346369fda5a09e646df0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ee02b1aca346369fda5a09e646df0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39947 } }
I20260926 15:07:55.858197 30980 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5ee02b1aca346369fda5a09e646df0e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:07:55.860332 30992 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5ee02b1aca346369fda5a09e646df0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5ee02b1aca346369fda5a09e646df0e. Latest consensus state: current_term: 1 leader_uuid: "f5ee02b1aca346369fda5a09e646df0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ee02b1aca346369fda5a09e646df0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39947 } } }
I20260926 15:07:55.860313 30991 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5ee02b1aca346369fda5a09e646df0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5ee02b1aca346369fda5a09e646df0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ee02b1aca346369fda5a09e646df0e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39947 } } }
I20260926 15:07:55.860769 30991 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5ee02b1aca346369fda5a09e646df0e [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:55.860769 30992 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5ee02b1aca346369fda5a09e646df0e [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:55.862826 30997 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:07:55.868306 30997 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:07:55.877633 30997 catalog_manager.cc:1347] Generated new cluster ID: 6174572746e24150bb61169b150558ba
I20260926 15:07:55.877761 30997 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:07:55.896418 30997 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:07:55.897830 30997 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:07:55.912649 30997 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5ee02b1aca346369fda5a09e646df0e: Generated new TSK 0
I20260926 15:07:55.913895 30997 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:07:55.928231 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 30913
W20260926 15:07:55.929453 30975 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.17:34821
W20260926 15:07:55.930226 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 30913: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 15:07:55.930362 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 30913
W20260926 15:07:55.943974 30898 connection.cc:535] client connection to 127.0.0.1:39947 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39947): Connection reset by peer (error 104)
2026-09-26T15:07:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1820 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:07:56 dist-test-slave-7153 krb5kdc[31011](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:07:56 dist-test-slave-7153 krb5kdc[31011](info): set up 2 sockets
Sep 26 15:07:56 dist-test-slave-7153 krb5kdc[31011](info): commencing operation
krb5kdc: starting...
W20260926 15:07:56.957618 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.995s user 0.006s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:07:57 dist-test-slave-7153 krb5kdc[31011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:07:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:07:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 15:07:57.209452 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34713
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:07:57.540088 31024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:57.540426 31024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:57.540560 31024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:57.547914 31024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:07:57.548043 31024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:57.548125 31024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:57.548218 31024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:07:57.548276 31024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:07:57.558543 31024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:57.559899 31024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:57.561256 31024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:57.574033 31029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:57.573987 31032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:57.574010 31030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:57.574746 31024 server_base.cc:1034] running on GCE node
I20260926 15:07:57.575618 31024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:57.577188 31024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:57.578444 31024 hybrid_clock.cc:648] HybridClock initialized: now 1790435277578385 us; error 51 us; skew 500 ppm
I20260926 15:07:57.578857 31024 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:57 dist-test-slave-7153 krb5kdc[31011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435277, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:57.587226 31024 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 15:07:57.589170 31024 webserver.cc:460] Webserver started at http://127.0.0.1:34117/ using document root <none> and password file <none>
I20260926 15:07:57.589941 31024 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:57.590068 31024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:57.590340 31024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:57.593672 31024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "7e66832da5914f2f8aaee989a4ae86fa"
format_stamp: "Formatted at 2026-09-26 15:07:57 on dist-test-slave-7153"
server_key: "81ec8d510817ff5c001094d0076bb631"
server_key_iv: "6b8dd19b5128dc916404ca5e6028e695"
server_key_version: "encryptionkey@0"
I20260926 15:07:57.594450 31024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "7e66832da5914f2f8aaee989a4ae86fa"
format_stamp: "Formatted at 2026-09-26 15:07:57 on dist-test-slave-7153"
server_key: "81ec8d510817ff5c001094d0076bb631"
server_key_iv: "6b8dd19b5128dc916404ca5e6028e695"
server_key_version: "encryptionkey@0"
I20260926 15:07:57.600694 31024 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260926 15:07:57.604741 31039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:57.606575 31024 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260926 15:07:57.606801 31024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "7e66832da5914f2f8aaee989a4ae86fa"
format_stamp: "Formatted at 2026-09-26 15:07:57 on dist-test-slave-7153"
server_key: "81ec8d510817ff5c001094d0076bb631"
server_key_iv: "6b8dd19b5128dc916404ca5e6028e695"
server_key_version: "encryptionkey@0"
I20260926 15:07:57.607021 31024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:57.613119 31024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:57.617859 31024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:57.618134 31024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:57.636929 31024 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34713
I20260926 15:07:57.636953 31090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34713 every 8 connection(s)
I20260926 15:07:57.638378 31024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:07:57.642194 31091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:57.644322 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 31024
I20260926 15:07:57.644506 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:07:57.645010 9709 external_mini_cluster.cc:1428] Setting key abc6a77b223dd5762a3abefa2d419c1b
I20260926 15:07:57.653035 31094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:57.646812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.17:47391 (local address 127.0.0.1:34713)
0926 15:07:57.647624 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:57.647638 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:57.647685 (+ 47us) server_negotiation.cc:407] Connection header received
0926 15:07:57.649529 (+ 1844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:57.649557 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:57.649945 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:57.650289 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:57.651373 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:57.651379 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:57.651392 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:57.651407 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:57.651469 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:57.651493 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:57.651678 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:57.651777 (+ 99us) server_negotiation.cc:299] Negotiation successful
0926 15:07:57.652029 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":336,"threads_started":1}
I20260926 15:07:57.654479 31091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e66832da5914f2f8aaee989a4ae86fa: Bootstrap starting.
I20260926 15:07:57.656824 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:07:57.657938 31091 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e66832da5914f2f8aaee989a4ae86fa: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:57.659299 31091 log.cc:824] T 00000000000000000000000000000000 P 7e66832da5914f2f8aaee989a4ae86fa: Log is configured to *not* fsync() on all Append() calls
I20260926 15:07:57.662576 31091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e66832da5914f2f8aaee989a4ae86fa: No bootstrap required, opened a new log
I20260926 15:07:57.664441 31094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:57.661892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.17:44869 (local address 127.0.0.1:34713)
0926 15:07:57.662097 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:57.662104 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:57.662518 (+ 414us) server_negotiation.cc:407] Connection header received
0926 15:07:57.662708 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:57.662714 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:57.662769 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:57.662850 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:57.663786 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:57.663790 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:57.663794 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:07:57.663808 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:57.663853 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:07:57.663870 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:57.663956 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:57.664095 (+ 139us) server_negotiation.cc:299] Negotiation successful
0926 15:07:57.664143 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260926 15:07:57.670675 31091 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e66832da5914f2f8aaee989a4ae86fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e66832da5914f2f8aaee989a4ae86fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34713 } }
I20260926 15:07:57.671103 31091 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e66832da5914f2f8aaee989a4ae86fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:57.671200 31091 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e66832da5914f2f8aaee989a4ae86fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e66832da5914f2f8aaee989a4ae86fa, State: Initialized, Role: FOLLOWER
I20260926 15:07:57.671847 31091 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e66832da5914f2f8aaee989a4ae86fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e66832da5914f2f8aaee989a4ae86fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34713 } }
I20260926 15:07:57.672115 31091 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e66832da5914f2f8aaee989a4ae86fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:07:57.672269 31091 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e66832da5914f2f8aaee989a4ae86fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:07:57.672425 31091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e66832da5914f2f8aaee989a4ae86fa [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:57.674747 31091 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e66832da5914f2f8aaee989a4ae86fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e66832da5914f2f8aaee989a4ae86fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34713 } }
I20260926 15:07:57.675290 31091 leader_election.cc:302] T 00000000000000000000000000000000 P 7e66832da5914f2f8aaee989a4ae86fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e66832da5914f2f8aaee989a4ae86fa; no voters:
I20260926 15:07:57.675879 31091 leader_election.cc:288] T 00000000000000000000000000000000 P 7e66832da5914f2f8aaee989a4ae86fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:07:57.676017 31101 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e66832da5914f2f8aaee989a4ae86fa [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:07:57.677060 31101 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e66832da5914f2f8aaee989a4ae86fa [term 1 LEADER]: Becoming Leader. State: Replica: 7e66832da5914f2f8aaee989a4ae86fa, State: Running, Role: LEADER
I20260926 15:07:57.677637 31101 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e66832da5914f2f8aaee989a4ae86fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e66832da5914f2f8aaee989a4ae86fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34713 } }
I20260926 15:07:57.678078 31091 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e66832da5914f2f8aaee989a4ae86fa [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:07:57.680397 31102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e66832da5914f2f8aaee989a4ae86fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e66832da5914f2f8aaee989a4ae86fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e66832da5914f2f8aaee989a4ae86fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34713 } } }
I20260926 15:07:57.680474 31103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e66832da5914f2f8aaee989a4ae86fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e66832da5914f2f8aaee989a4ae86fa. Latest consensus state: current_term: 1 leader_uuid: "7e66832da5914f2f8aaee989a4ae86fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e66832da5914f2f8aaee989a4ae86fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34713 } } }
I20260926 15:07:57.680830 31102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e66832da5914f2f8aaee989a4ae86fa [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:57.680828 31103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e66832da5914f2f8aaee989a4ae86fa [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:57.682786 31107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:07:57.688977 31107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:07:57.698283 31107 catalog_manager.cc:1347] Generated new cluster ID: df97d04e7c3746c4a3690d6a1684ec28
I20260926 15:07:57.698401 31107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:07:57.713130 31107 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:07:57.713579 31107 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:07:57.720974 31107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e66832da5914f2f8aaee989a4ae86fa: Generated new TSK 0
I20260926 15:07:57.721848 31107 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:07:57.728847 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 31024
W20260926 15:07:57.730150 31086 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.17:47391
W20260926 15:07:57.730924 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 31024: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 15:07:57.731050 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 31024
2026-09-26T15:07:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1800 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:07:57 dist-test-slave-7153 krb5kdc[31122](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:07:57 dist-test-slave-7153 krb5kdc[31122](info): set up 2 sockets
Sep 26 15:07:57 dist-test-slave-7153 krb5kdc[31122](info): commencing operation
krb5kdc: starting...
W20260926 15:07:58.757730 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.994s user 0.004s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:07:58 dist-test-slave-7153 krb5kdc[31122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:07:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:07:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 15:07:59.020274 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35729
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:07:59.352532 31135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:07:59.352833 31135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:07:59.352934 31135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:07:59.360226 31135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:07:59.360354 31135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:07:59.360404 31135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:07:59.360463 31135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:07:59.360507 31135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:07:59.370905 31135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35729
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:07:59.372234 31135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:07:59.373545 31135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:07:59.385550 31143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:59.385694 31140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:07:59.385601 31141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:07:59.386121 31135 server_base.cc:1034] running on GCE node
I20260926 15:07:59.387030 31135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:07:59.388516 31135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:07:59.389745 31135 hybrid_clock.cc:648] HybridClock initialized: now 1790435279389685 us; error 53 us; skew 500 ppm
I20260926 15:07:59.390130 31135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:07:59 dist-test-slave-7153 krb5kdc[31122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435279, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:07:59.393296 31135 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 15:07:59.394865 31135 webserver.cc:460] Webserver started at http://127.0.0.1:37755/ using document root <none> and password file <none>
I20260926 15:07:59.395571 31135 fs_manager.cc:359] Metadata directory not provided
I20260926 15:07:59.395690 31135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:07:59.395985 31135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:07:59.399073 31135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "f7eb5290dd1340e7b543924d7bf7ee9b"
format_stamp: "Formatted at 2026-09-26 15:07:59 on dist-test-slave-7153"
server_key: "439b073c756e074c1c4fe5af6ea56f66"
server_key_iv: "4a225096758740505e63616c32be5ae4"
server_key_version: "encryptionkey@0"
I20260926 15:07:59.399886 31135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "f7eb5290dd1340e7b543924d7bf7ee9b"
format_stamp: "Formatted at 2026-09-26 15:07:59 on dist-test-slave-7153"
server_key: "439b073c756e074c1c4fe5af6ea56f66"
server_key_iv: "4a225096758740505e63616c32be5ae4"
server_key_version: "encryptionkey@0"
I20260926 15:07:59.405458 31135 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260926 15:07:59.409200 31150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:07:59.410614 31135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:07:59.410810 31135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "f7eb5290dd1340e7b543924d7bf7ee9b"
format_stamp: "Formatted at 2026-09-26 15:07:59 on dist-test-slave-7153"
server_key: "439b073c756e074c1c4fe5af6ea56f66"
server_key_iv: "4a225096758740505e63616c32be5ae4"
server_key_version: "encryptionkey@0"
I20260926 15:07:59.411029 31135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:07:59.433691 31135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:07:59.437980 31135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:07:59.438261 31135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:07:59.456264 31135 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35729
I20260926 15:07:59.456305 31201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35729 every 8 connection(s)
I20260926 15:07:59.457733 31135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:07:59.461700 31202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:07:59.464764 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 31135
I20260926 15:07:59.464937 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:07:59.465493 9709 external_mini_cluster.cc:1428] Setting key 69b12d165f442d663665cf85448f454c
I20260926 15:07:59.476032 31202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7eb5290dd1340e7b543924d7bf7ee9b: Bootstrap starting.
I20260926 15:07:59.479879 31202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7eb5290dd1340e7b543924d7bf7ee9b: Neither blocks nor log segments found. Creating new log.
I20260926 15:07:59.481328 31202 log.cc:824] T 00000000000000000000000000000000 P f7eb5290dd1340e7b543924d7bf7ee9b: Log is configured to *not* fsync() on all Append() calls
Sep 26 15:07:59 dist-test-slave-7153 krb5kdc[31122](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260926 15:07:59.485314 31202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7eb5290dd1340e7b543924d7bf7ee9b: No bootstrap required, opened a new log
I20260926 15:07:59.492974 31205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:59.467562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.17:59347 (local address 127.0.0.1:35729)
0926 15:07:59.468492 (+ 930us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:59.468509 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:59.469679 (+ 1170us) server_negotiation.cc:407] Connection header received
0926 15:07:59.471569 (+ 1890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:59.471603 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:59.472240 (+ 637us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:59.472727 (+ 487us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:59.474125 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:59.476134 (+ 2009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:59.477510 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:59.478574 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:59.478757 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:59.483949 (+ 5192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:59.483959 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:59.483979 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:59.484020 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:59.487815 (+ 3795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:59.489261 (+ 1446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:59.489270 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:59.489281 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:59.489493 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:59.490020 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:59.490024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:59.490026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:59.490623 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:59.490976 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:59.491510 (+ 534us) server_negotiation.cc:299] Negotiation successful
0926 15:07:59.491851 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":353,"threads_started":1}
I20260926 15:07:59.494560 31202 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7eb5290dd1340e7b543924d7bf7ee9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7eb5290dd1340e7b543924d7bf7ee9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35729 } }
I20260926 15:07:59.494958 31202 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7eb5290dd1340e7b543924d7bf7ee9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:07:59.495064 31202 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7eb5290dd1340e7b543924d7bf7ee9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7eb5290dd1340e7b543924d7bf7ee9b, State: Initialized, Role: FOLLOWER
I20260926 15:07:59.495680 31202 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7eb5290dd1340e7b543924d7bf7ee9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7eb5290dd1340e7b543924d7bf7ee9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35729 } }
I20260926 15:07:59.495965 31202 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7eb5290dd1340e7b543924d7bf7ee9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:07:59.496102 31202 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7eb5290dd1340e7b543924d7bf7ee9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:07:59.496276 31202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7eb5290dd1340e7b543924d7bf7ee9b [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:07:59.497066 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:07:59.499276 31202 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7eb5290dd1340e7b543924d7bf7ee9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7eb5290dd1340e7b543924d7bf7ee9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35729 } }
I20260926 15:07:59.499886 31202 leader_election.cc:302] T 00000000000000000000000000000000 P f7eb5290dd1340e7b543924d7bf7ee9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7eb5290dd1340e7b543924d7bf7ee9b; no voters:
I20260926 15:07:59.500803 31202 leader_election.cc:288] T 00000000000000000000000000000000 P f7eb5290dd1340e7b543924d7bf7ee9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:07:59.500958 31208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7eb5290dd1340e7b543924d7bf7ee9b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:07:59.501641 31208 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7eb5290dd1340e7b543924d7bf7ee9b [term 1 LEADER]: Becoming Leader. State: Replica: f7eb5290dd1340e7b543924d7bf7ee9b, State: Running, Role: LEADER
I20260926 15:07:59.502254 31208 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7eb5290dd1340e7b543924d7bf7ee9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7eb5290dd1340e7b543924d7bf7ee9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35729 } }
I20260926 15:07:59.504894 31202 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7eb5290dd1340e7b543924d7bf7ee9b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:07:59.506651 31212 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7eb5290dd1340e7b543924d7bf7ee9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7eb5290dd1340e7b543924d7bf7ee9b. Latest consensus state: current_term: 1 leader_uuid: "f7eb5290dd1340e7b543924d7bf7ee9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7eb5290dd1340e7b543924d7bf7ee9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35729 } } }
I20260926 15:07:59.506702 31213 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7eb5290dd1340e7b543924d7bf7ee9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7eb5290dd1340e7b543924d7bf7ee9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7eb5290dd1340e7b543924d7bf7ee9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35729 } } }
I20260926 15:07:59.507040 31212 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7eb5290dd1340e7b543924d7bf7ee9b [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:59.507038 31213 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7eb5290dd1340e7b543924d7bf7ee9b [sys.catalog]: This master's current role is: LEADER
I20260926 15:07:59.519011 31218 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:07:59.523012 31205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:07:59.502769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.17:58597 (local address 127.0.0.1:35729)
0926 15:07:59.502950 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 15:07:59.502956 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:07:59.503481 (+ 525us) server_negotiation.cc:407] Connection header received
0926 15:07:59.503755 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:07:59.503761 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:07:59.503849 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:07:59.503929 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:07:59.507078 (+ 3149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:59.508316 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:59.510446 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:07:59.511177 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:07:59.511284 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:07:59.515130 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:07:59.515139 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:07:59.515145 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:07:59.515200 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:07:59.519267 (+ 4067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:59.520395 (+ 1128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:59.520401 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:59.520403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:59.520552 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:07:59.521024 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:07:59.521027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:07:59.521030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:07:59.521493 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:07:59.521784 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0926 15:07:59.522425 (+ 641us) server_negotiation.cc:299] Negotiation successful
0926 15:07:59.522649 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 15:07:59.525933 31218 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:07:59.537669 31218 catalog_manager.cc:1347] Generated new cluster ID: b6a3894a09724e258fbcb80a133a0901
I20260926 15:07:59.537803 31218 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:07:59.548619 31218 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:07:59.550611 31218 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:07:59.568151 31218 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7eb5290dd1340e7b543924d7bf7ee9b: Generated new TSK 0
I20260926 15:07:59.569427 31218 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:07:59.577829 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 31135
I20260926 15:07:59.579270 31197 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:07:59.669858 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 31135
W20260926 15:07:59.683835 31120 connection.cc:535] client connection to 127.0.0.1:35729 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35729): Connection reset by peer (error 104)
2026-09-26T15:07:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1939 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:07:59 dist-test-slave-7153 krb5kdc[31234](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:07:59 dist-test-slave-7153 krb5kdc[31234](info): set up 2 sockets
Sep 26 15:07:59 dist-test-slave-7153 krb5kdc[31234](info): commencing operation
krb5kdc: starting...
W20260926 15:08:00.701473 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.997s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:08:00 dist-test-slave-7153 krb5kdc[31234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:08:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:08:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 15:08:00.956216 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41587
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:08:01.287905 31247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:08:01.288256 31247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:08:01.288350 31247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:08:01.295698 31247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:08:01.295831 31247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:08:01.295878 31247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:08:01.295938 31247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:08:01.295981 31247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:08:01.306149 31247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41587
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:08:01.307400 31247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:08:01.308709 31247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:08:01.320551 31252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:01.320633 31255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:01.320665 31253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:08:01.321350 31247 server_base.cc:1034] running on GCE node
I20260926 15:08:01.322144 31247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:08:01.323634 31247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:08:01.324837 31247 hybrid_clock.cc:648] HybridClock initialized: now 1790435281324778 us; error 57 us; skew 500 ppm
I20260926 15:08:01.325201 31247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:08:01 dist-test-slave-7153 krb5kdc[31234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435281, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:08:01.328223 31247 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 15:08:01.329757 31247 webserver.cc:460] Webserver started at http://127.0.0.1:43339/ using document root <none> and password file <none>
I20260926 15:08:01.330426 31247 fs_manager.cc:359] Metadata directory not provided
I20260926 15:08:01.330528 31247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:08:01.330787 31247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:08:01.334138 31247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "c0da2f145f99404e9b65ea40de6bcb7d"
format_stamp: "Formatted at 2026-09-26 15:08:01 on dist-test-slave-7153"
server_key: "75642a7c466ca910c42831f9920b21df"
server_key_iv: "56eac01196386af690b38e49f16573d3"
server_key_version: "encryptionkey@0"
I20260926 15:08:01.334885 31247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "c0da2f145f99404e9b65ea40de6bcb7d"
format_stamp: "Formatted at 2026-09-26 15:08:01 on dist-test-slave-7153"
server_key: "75642a7c466ca910c42831f9920b21df"
server_key_iv: "56eac01196386af690b38e49f16573d3"
server_key_version: "encryptionkey@0"
I20260926 15:08:01.340338 31247 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260926 15:08:01.344648 31262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:08:01.346240 31247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 15:08:01.346407 31247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "c0da2f145f99404e9b65ea40de6bcb7d"
format_stamp: "Formatted at 2026-09-26 15:08:01 on dist-test-slave-7153"
server_key: "75642a7c466ca910c42831f9920b21df"
server_key_iv: "56eac01196386af690b38e49f16573d3"
server_key_version: "encryptionkey@0"
I20260926 15:08:01.346601 31247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:08:01.365761 31247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:08:01.370004 31247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:08:01.370270 31247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:08:01.387876 31247 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41587
I20260926 15:08:01.387904 31313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41587 every 8 connection(s)
I20260926 15:08:01.389314 31247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:08:01.390856 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 31247
I20260926 15:08:01.391036 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:08:01.391561 9709 external_mini_cluster.cc:1428] Setting key 5f4e00566c46833aee021bd3b8210bf5
I20260926 15:08:01.393905 31314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:08:01.404975 31317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:01.393461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.17:43103 (local address 127.0.0.1:41587)
0926 15:08:01.394236 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:01.394249 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:01.394290 (+ 41us) server_negotiation.cc:407] Connection header received
0926 15:08:01.395898 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:01.395921 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:01.396350 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:01.396721 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:01.397365 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:01.398716 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:01.400539 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:01.401294 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:01.401420 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:01.402639 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:01.402647 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:01.402665 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:08:01.403256 (+ 591us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:01.403335 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:08:01.403360 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:01.403483 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:01.403762 (+ 279us) server_negotiation.cc:299] Negotiation successful
0926 15:08:01.403958 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":347,"threads_started":1}
I20260926 15:08:01.406424 31314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0da2f145f99404e9b65ea40de6bcb7d: Bootstrap starting.
I20260926 15:08:01.408895 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:08:01.409803 31314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0da2f145f99404e9b65ea40de6bcb7d: Neither blocks nor log segments found. Creating new log.
I20260926 15:08:01.410854 31314 log.cc:824] T 00000000000000000000000000000000 P c0da2f145f99404e9b65ea40de6bcb7d: Log is configured to *not* fsync() on all Append() calls
I20260926 15:08:01.413923 31314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0da2f145f99404e9b65ea40de6bcb7d: No bootstrap required, opened a new log
I20260926 15:08:01.420009 31317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:01.413315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.17:37645 (local address 127.0.0.1:41587)
0926 15:08:01.413505 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:01.413511 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:01.413852 (+ 341us) server_negotiation.cc:407] Connection header received
0926 15:08:01.414061 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:01.414067 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:01.414132 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:01.414244 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:01.414865 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:01.415658 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:01.417511 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:01.417953 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:01.418050 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:01.419276 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:01.419281 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:01.419287 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:08:01.419306 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:01.419345 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:08:01.419363 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:01.419448 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:01.419635 (+ 187us) server_negotiation.cc:299] Negotiation successful
0926 15:08:01.419683 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 15:08:01.423226 31314 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0da2f145f99404e9b65ea40de6bcb7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0da2f145f99404e9b65ea40de6bcb7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41587 } }
I20260926 15:08:01.423611 31314 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0da2f145f99404e9b65ea40de6bcb7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:08:01.423686 31314 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0da2f145f99404e9b65ea40de6bcb7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0da2f145f99404e9b65ea40de6bcb7d, State: Initialized, Role: FOLLOWER
I20260926 15:08:01.424273 31314 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0da2f145f99404e9b65ea40de6bcb7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0da2f145f99404e9b65ea40de6bcb7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41587 } }
I20260926 15:08:01.424526 31314 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0da2f145f99404e9b65ea40de6bcb7d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:08:01.424681 31314 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0da2f145f99404e9b65ea40de6bcb7d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:08:01.424863 31314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0da2f145f99404e9b65ea40de6bcb7d [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:08:01.427034 31314 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0da2f145f99404e9b65ea40de6bcb7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0da2f145f99404e9b65ea40de6bcb7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41587 } }
I20260926 15:08:01.427597 31314 leader_election.cc:302] T 00000000000000000000000000000000 P c0da2f145f99404e9b65ea40de6bcb7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0da2f145f99404e9b65ea40de6bcb7d; no voters:
I20260926 15:08:01.428218 31314 leader_election.cc:288] T 00000000000000000000000000000000 P c0da2f145f99404e9b65ea40de6bcb7d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:08:01.428371 31324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0da2f145f99404e9b65ea40de6bcb7d [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:08:01.428998 31324 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0da2f145f99404e9b65ea40de6bcb7d [term 1 LEADER]: Becoming Leader. State: Replica: c0da2f145f99404e9b65ea40de6bcb7d, State: Running, Role: LEADER
I20260926 15:08:01.429529 31324 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0da2f145f99404e9b65ea40de6bcb7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0da2f145f99404e9b65ea40de6bcb7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41587 } }
I20260926 15:08:01.430226 31314 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0da2f145f99404e9b65ea40de6bcb7d [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:08:01.432029 31325 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0da2f145f99404e9b65ea40de6bcb7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0da2f145f99404e9b65ea40de6bcb7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0da2f145f99404e9b65ea40de6bcb7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41587 } } }
I20260926 15:08:01.432422 31325 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0da2f145f99404e9b65ea40de6bcb7d [sys.catalog]: This master's current role is: LEADER
I20260926 15:08:01.432857 31326 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0da2f145f99404e9b65ea40de6bcb7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0da2f145f99404e9b65ea40de6bcb7d. Latest consensus state: current_term: 1 leader_uuid: "c0da2f145f99404e9b65ea40de6bcb7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0da2f145f99404e9b65ea40de6bcb7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41587 } } }
I20260926 15:08:01.433185 31326 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0da2f145f99404e9b65ea40de6bcb7d [sys.catalog]: This master's current role is: LEADER
I20260926 15:08:01.433998 31332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:08:01.438786 31332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:08:01.447775 31332 catalog_manager.cc:1347] Generated new cluster ID: b5ad4b13a7e248eeaf8dc9e4924fa906
I20260926 15:08:01.447901 31332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:08:01.458884 31332 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:08:01.460764 31332 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:08:01.470726 31332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0da2f145f99404e9b65ea40de6bcb7d: Generated new TSK 0
I20260926 15:08:01.471813 31332 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:08:01.490828 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 31247
W20260926 15:08:01.491961 31309 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.17:43103
W20260926 15:08:01.492684 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 31247: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 15:08:01.492832 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 31247
W20260926 15:08:01.506352 31232 connection.cc:535] client connection to 127.0.0.1:41587 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41587): Connection reset by peer (error 104)
2026-09-26T15:08:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1821 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:08:01 dist-test-slave-7153 krb5kdc[31345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:08:01 dist-test-slave-7153 krb5kdc[31345](info): set up 2 sockets
Sep 26 15:08:01 dist-test-slave-7153 krb5kdc[31345](info): commencing operation
krb5kdc: starting...
W20260926 15:08:02.517434 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.991s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:08:02 dist-test-slave-7153 krb5kdc[31345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:08:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:08:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260926 15:08:02.765460 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34141
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:08:03.093891 31358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:08:03.094211 31358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:08:03.094306 31358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:08:03.101768 31358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:08:03.101905 31358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:08:03.101953 31358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:08:03.102016 31358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:08:03.102061 31358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:08:03.112334 31358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34141
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:08:03.113739 31358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:08:03.115125 31358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:08:03.127149 31363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:03.127183 31364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:03.127074 31366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:08:03.128139 31358 server_base.cc:1034] running on GCE node
I20260926 15:08:03.128960 31358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:08:03.130558 31358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:08:03.131784 31358 hybrid_clock.cc:648] HybridClock initialized: now 1790435283131738 us; error 44 us; skew 500 ppm
I20260926 15:08:03.132182 31358 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:08:03 dist-test-slave-7153 krb5kdc[31345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435283, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:08:03.141278 31358 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260926 15:08:03.142963 31358 webserver.cc:460] Webserver started at http://127.0.0.1:43053/ using document root <none> and password file <none>
I20260926 15:08:03.143700 31358 fs_manager.cc:359] Metadata directory not provided
I20260926 15:08:03.143827 31358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:08:03.144140 31358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:08:03.147308 31358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "9f66ec8de87e4a129b5f42879d9a6969"
format_stamp: "Formatted at 2026-09-26 15:08:03 on dist-test-slave-7153"
server_key: "de2e3f0815cb8e72ecedbc6a782f28c1"
server_key_iv: "0be227a4e0d8371d00e1a4315c5ba24b"
server_key_version: "encryptionkey@0"
I20260926 15:08:03.148120 31358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "9f66ec8de87e4a129b5f42879d9a6969"
format_stamp: "Formatted at 2026-09-26 15:08:03 on dist-test-slave-7153"
server_key: "de2e3f0815cb8e72ecedbc6a782f28c1"
server_key_iv: "0be227a4e0d8371d00e1a4315c5ba24b"
server_key_version: "encryptionkey@0"
I20260926 15:08:03.153532 31358 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260926 15:08:03.157333 31373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:08:03.158846 31358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:08:03.159049 31358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "9f66ec8de87e4a129b5f42879d9a6969"
format_stamp: "Formatted at 2026-09-26 15:08:03 on dist-test-slave-7153"
server_key: "de2e3f0815cb8e72ecedbc6a782f28c1"
server_key_iv: "0be227a4e0d8371d00e1a4315c5ba24b"
server_key_version: "encryptionkey@0"
I20260926 15:08:03.159250 31358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:08:03.165191 31358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:08:03.169427 31358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:08:03.169724 31358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:08:03.187934 31358 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34141
I20260926 15:08:03.187975 31424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34141 every 8 connection(s)
I20260926 15:08:03.189637 31358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:08:03.193434 31425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:08:03.199389 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 31358
I20260926 15:08:03.199587 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:08:03.200143 9709 external_mini_cluster.cc:1428] Setting key f40415223fe1a458c6c79640520502eb
I20260926 15:08:03.205219 31425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f66ec8de87e4a129b5f42879d9a6969: Bootstrap starting.
I20260926 15:08:03.208475 31428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:03.202128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.17:52973 (local address 127.0.0.1:34141)
0926 15:08:03.203044 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:03.203057 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:03.203107 (+ 50us) server_negotiation.cc:407] Connection header received
0926 15:08:03.204909 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:03.204941 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:03.205416 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:03.205780 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:03.206856 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:03.206863 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:03.206877 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:08:03.206891 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:03.207003 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:08:03.207026 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:03.207201 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:03.207312 (+ 111us) server_negotiation.cc:299] Negotiation successful
0926 15:08:03.207513 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":404,"threads_started":1}
I20260926 15:08:03.208856 31425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f66ec8de87e4a129b5f42879d9a6969: Neither blocks nor log segments found. Creating new log.
I20260926 15:08:03.209961 31425 log.cc:824] T 00000000000000000000000000000000 P 9f66ec8de87e4a129b5f42879d9a6969: Log is configured to *not* fsync() on all Append() calls
I20260926 15:08:03.212422 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:08:03.213220 31425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f66ec8de87e4a129b5f42879d9a6969: No bootstrap required, opened a new log
I20260926 15:08:03.221366 31428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:03.217709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.17:49201 (local address 127.0.0.1:34141)
0926 15:08:03.217982 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:03.217990 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:03.219116 (+ 1126us) server_negotiation.cc:407] Connection header received
0926 15:08:03.219345 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:03.219351 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:03.219425 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:03.219561 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:03.220575 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:03.220579 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:03.220583 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:08:03.220603 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:03.220651 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:08:03.220671 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:03.220774 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:03.220868 (+ 94us) server_negotiation.cc:299] Negotiation successful
0926 15:08:03.220920 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260926 15:08:03.223198 31425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f66ec8de87e4a129b5f42879d9a6969 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f66ec8de87e4a129b5f42879d9a6969" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34141 } }
I20260926 15:08:03.223585 31425 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f66ec8de87e4a129b5f42879d9a6969 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:08:03.223693 31425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f66ec8de87e4a129b5f42879d9a6969 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f66ec8de87e4a129b5f42879d9a6969, State: Initialized, Role: FOLLOWER
I20260926 15:08:03.224356 31425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f66ec8de87e4a129b5f42879d9a6969 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f66ec8de87e4a129b5f42879d9a6969" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34141 } }
I20260926 15:08:03.224651 31425 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f66ec8de87e4a129b5f42879d9a6969 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:08:03.224804 31425 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f66ec8de87e4a129b5f42879d9a6969 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:08:03.224964 31425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f66ec8de87e4a129b5f42879d9a6969 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:08:03.227240 31425 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f66ec8de87e4a129b5f42879d9a6969 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f66ec8de87e4a129b5f42879d9a6969" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34141 } }
I20260926 15:08:03.227774 31425 leader_election.cc:302] T 00000000000000000000000000000000 P 9f66ec8de87e4a129b5f42879d9a6969 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f66ec8de87e4a129b5f42879d9a6969; no voters:
I20260926 15:08:03.228425 31425 leader_election.cc:288] T 00000000000000000000000000000000 P 9f66ec8de87e4a129b5f42879d9a6969 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:08:03.228606 31435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f66ec8de87e4a129b5f42879d9a6969 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:08:03.229305 31435 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f66ec8de87e4a129b5f42879d9a6969 [term 1 LEADER]: Becoming Leader. State: Replica: 9f66ec8de87e4a129b5f42879d9a6969, State: Running, Role: LEADER
I20260926 15:08:03.229976 31435 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f66ec8de87e4a129b5f42879d9a6969 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f66ec8de87e4a129b5f42879d9a6969" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34141 } }
I20260926 15:08:03.230969 31425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f66ec8de87e4a129b5f42879d9a6969 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:08:03.232862 31436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f66ec8de87e4a129b5f42879d9a6969 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f66ec8de87e4a129b5f42879d9a6969. Latest consensus state: current_term: 1 leader_uuid: "9f66ec8de87e4a129b5f42879d9a6969" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f66ec8de87e4a129b5f42879d9a6969" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34141 } } }
I20260926 15:08:03.233369 31436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f66ec8de87e4a129b5f42879d9a6969 [sys.catalog]: This master's current role is: LEADER
I20260926 15:08:03.233276 31437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f66ec8de87e4a129b5f42879d9a6969 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f66ec8de87e4a129b5f42879d9a6969" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f66ec8de87e4a129b5f42879d9a6969" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34141 } } }
I20260926 15:08:03.233598 31437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f66ec8de87e4a129b5f42879d9a6969 [sys.catalog]: This master's current role is: LEADER
I20260926 15:08:03.235633 31442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:08:03.241797 31442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:08:03.251175 31442 catalog_manager.cc:1347] Generated new cluster ID: 53c874888d7c406e9aa60e29e9d60737
I20260926 15:08:03.251298 31442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:08:03.267200 31442 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:08:03.267686 31442 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:08:03.277334 31442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f66ec8de87e4a129b5f42879d9a6969: Generated new TSK 0
I20260926 15:08:03.278173 31442 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:08:03.283514 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 31358
W20260926 15:08:03.285352 31420 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.17:52973
W20260926 15:08:03.286073 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 31358: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 15:08:03.286206 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 31358
2026-09-26T15:08:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1793 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:08:03 dist-test-slave-7153 krb5kdc[31456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:08:03 dist-test-slave-7153 krb5kdc[31456](info): set up 2 sockets
Sep 26 15:08:03 dist-test-slave-7153 krb5kdc[31456](info): commencing operation
krb5kdc: starting...
W20260926 15:08:04.309459 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.991s user 0.004s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:08:04 dist-test-slave-7153 krb5kdc[31456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:08:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:08:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:08:04.581306 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:46447
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:37993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:46447
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:08:04.912122 31469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:08:04.912415 31469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:08:04.912509 31469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:08:04.920050 31469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:08:04.920202 31469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:08:04.920277 31469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:08:04.920360 31469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:08:04.920414 31469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:08:04.930488 31469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:37993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:46447
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:46447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:08:04.931711 31469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:08:04.932971 31469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:08:04.944473 31474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:04.944559 31475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:04.944499 31477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:08:04.950872 31469 server_base.cc:1034] running on GCE node
I20260926 15:08:04.951803 31469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:08:04.953481 31469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:08:04.954718 31469 hybrid_clock.cc:648] HybridClock initialized: now 1790435284954671 us; error 45 us; skew 500 ppm
I20260926 15:08:04.955150 31469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:08:04 dist-test-slave-7153 krb5kdc[31456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435284, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:08:04.958446 31469 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:08:04.960047 31469 webserver.cc:460] Webserver started at http://127.9.123.126:46527/ using document root <none> and password file <none>
I20260926 15:08:04.960752 31469 fs_manager.cc:359] Metadata directory not provided
I20260926 15:08:04.960868 31469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:08:04.961258 31469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:08:04.964359 31469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "761a6d1952df44c8b908ce4eaed17467"
format_stamp: "Formatted at 2026-09-26 15:08:04 on dist-test-slave-7153"
server_key: "aed49474bff0d63013f9db0ed5ffad36"
server_key_iv: "ee5c98e7143898ed31340a0a3aaad89d"
server_key_version: "encryptionkey@0"
I20260926 15:08:04.965165 31469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "761a6d1952df44c8b908ce4eaed17467"
format_stamp: "Formatted at 2026-09-26 15:08:04 on dist-test-slave-7153"
server_key: "aed49474bff0d63013f9db0ed5ffad36"
server_key_iv: "ee5c98e7143898ed31340a0a3aaad89d"
server_key_version: "encryptionkey@0"
I20260926 15:08:04.970490 31469 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260926 15:08:04.974218 31484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:08:04.975795 31469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:08:04.975975 31469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "761a6d1952df44c8b908ce4eaed17467"
format_stamp: "Formatted at 2026-09-26 15:08:04 on dist-test-slave-7153"
server_key: "aed49474bff0d63013f9db0ed5ffad36"
server_key_iv: "ee5c98e7143898ed31340a0a3aaad89d"
server_key_version: "encryptionkey@0"
I20260926 15:08:04.976168 31469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:08:04.992916 31469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:08:04.997334 31469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:08:04.997618 31469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:08:05.015154 31469 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:46447
I20260926 15:08:05.015199 31535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:46447 every 8 connection(s)
I20260926 15:08:05.016498 31469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:08:05.020418 31536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:08:05.025658 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 31469
I20260926 15:08:05.025864 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:08:05.026410 9709 external_mini_cluster.cc:1428] Setting key 84febe5e95dafc1a39d3f124ffd5871c
I20260926 15:08:05.032311 31536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 761a6d1952df44c8b908ce4eaed17467: Bootstrap starting.
I20260926 15:08:05.035552 31536 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 761a6d1952df44c8b908ce4eaed17467: Neither blocks nor log segments found. Creating new log.
I20260926 15:08:05.036646 31536 log.cc:824] T 00000000000000000000000000000000 P 761a6d1952df44c8b908ce4eaed17467: Log is configured to *not* fsync() on all Append() calls
I20260926 15:08:05.039501 31536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 761a6d1952df44c8b908ce4eaed17467: No bootstrap required, opened a new log
Sep 26 15:08:05 dist-test-slave-7153 krb5kdc[31456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:08:05.047499 31536 raft_consensus.cc:348] T 00000000000000000000000000000000 P 761a6d1952df44c8b908ce4eaed17467 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "761a6d1952df44c8b908ce4eaed17467" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 46447 } }
I20260926 15:08:05.047956 31536 raft_consensus.cc:374] T 00000000000000000000000000000000 P 761a6d1952df44c8b908ce4eaed17467 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:08:05.048063 31536 raft_consensus.cc:729] T 00000000000000000000000000000000 P 761a6d1952df44c8b908ce4eaed17467 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 761a6d1952df44c8b908ce4eaed17467, State: Initialized, Role: FOLLOWER
I20260926 15:08:05.048659 31536 consensus_queue.cc:260] T 00000000000000000000000000000000 P 761a6d1952df44c8b908ce4eaed17467 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "761a6d1952df44c8b908ce4eaed17467" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 46447 } }
I20260926 15:08:05.048897 31536 raft_consensus.cc:388] T 00000000000000000000000000000000 P 761a6d1952df44c8b908ce4eaed17467 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:08:05.049024 31536 raft_consensus.cc:482] T 00000000000000000000000000000000 P 761a6d1952df44c8b908ce4eaed17467 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:08:05.049202 31536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 761a6d1952df44c8b908ce4eaed17467 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:08:05.049398 31539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:05.028426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48277 (local address 127.9.123.126:46447)
0926 15:08:05.029332 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:05.029344 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:05.029385 (+ 41us) server_negotiation.cc:407] Connection header received
0926 15:08:05.031172 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:05.031200 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:05.031620 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:05.032017 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:05.033483 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:05.034827 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:05.036145 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:05.036868 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:05.037046 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:05.041244 (+ 4198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:05.041255 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:05.041272 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:05.041317 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:05.044703 (+ 3386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:05.045662 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:05.045674 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:05.045681 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:05.045868 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:05.046346 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:05.046350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:05.046352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:05.046962 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:05.047273 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:05.047781 (+ 508us) server_negotiation.cc:299] Negotiation successful
0926 15:08:05.048193 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":416,"threads_started":1}
I20260926 15:08:05.052006 31536 raft_consensus.cc:504] T 00000000000000000000000000000000 P 761a6d1952df44c8b908ce4eaed17467 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "761a6d1952df44c8b908ce4eaed17467" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 46447 } }
I20260926 15:08:05.052506 31536 leader_election.cc:302] T 00000000000000000000000000000000 P 761a6d1952df44c8b908ce4eaed17467 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 761a6d1952df44c8b908ce4eaed17467; no voters:
I20260926 15:08:05.053179 31536 leader_election.cc:288] T 00000000000000000000000000000000 P 761a6d1952df44c8b908ce4eaed17467 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:08:05.053318 31541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 761a6d1952df44c8b908ce4eaed17467 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:08:05.053547 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:08:05.055480 31536 sys_catalog.cc:564] T 00000000000000000000000000000000 P 761a6d1952df44c8b908ce4eaed17467 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:08:05.061730 31541 raft_consensus.cc:686] T 00000000000000000000000000000000 P 761a6d1952df44c8b908ce4eaed17467 [term 1 LEADER]: Becoming Leader. State: Replica: 761a6d1952df44c8b908ce4eaed17467, State: Running, Role: LEADER
I20260926 15:08:05.062647 31541 consensus_queue.cc:237] T 00000000000000000000000000000000 P 761a6d1952df44c8b908ce4eaed17467 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "761a6d1952df44c8b908ce4eaed17467" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 46447 } }
I20260926 15:08:05.066002 31543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 761a6d1952df44c8b908ce4eaed17467 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 761a6d1952df44c8b908ce4eaed17467. Latest consensus state: current_term: 1 leader_uuid: "761a6d1952df44c8b908ce4eaed17467" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "761a6d1952df44c8b908ce4eaed17467" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 46447 } } }
I20260926 15:08:05.066448 31543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 761a6d1952df44c8b908ce4eaed17467 [sys.catalog]: This master's current role is: LEADER
I20260926 15:08:05.067019 31542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 761a6d1952df44c8b908ce4eaed17467 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "761a6d1952df44c8b908ce4eaed17467" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "761a6d1952df44c8b908ce4eaed17467" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 46447 } } }
I20260926 15:08:05.067330 31542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 761a6d1952df44c8b908ce4eaed17467 [sys.catalog]: This master's current role is: LEADER
I20260926 15:08:05.067332 31556 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:08:05.077353 31556 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:08:05.091089 31539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:05.063875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55407 (local address 127.9.123.126:46447)
0926 15:08:05.064060 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:05.064067 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:05.068126 (+ 4059us) server_negotiation.cc:407] Connection header received
0926 15:08:05.068443 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:05.068449 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:05.068531 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:05.068607 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:05.070668 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:05.071882 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:05.073726 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:05.074398 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:05.074520 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:05.078011 (+ 3491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:05.078018 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:05.078023 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:05.078072 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:05.086409 (+ 8337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:05.087680 (+ 1271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:05.087686 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:05.087691 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:05.087891 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:05.088502 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:05.088507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:05.088511 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:05.088959 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:05.089265 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:05.090671 (+ 1406us) server_negotiation.cc:299] Negotiation successful
0926 15:08:05.090816 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 15:08:05.093199 31556 catalog_manager.cc:1347] Generated new cluster ID: 560e060e3c5047ba917febdc4a163135
I20260926 15:08:05.093473 31556 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:08:05.113348 31556 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:08:05.115000 31556 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:08:05.122159 31556 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 761a6d1952df44c8b908ce4eaed17467: Generated new TSK 0
I20260926 15:08:05.123116 31556 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:08:05.129810 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 31469
I20260926 15:08:05.130909 31531 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:08:05.221588 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 31469
W20260926 15:08:05.236907 31454 connection.cc:535] client connection to 127.9.123.126:46447 recv error: Network error: failed to read from TLS socket (remote: 127.9.123.126:46447): Connection reset by peer (error 104)
2026-09-26T15:08:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1937 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:08:05 dist-test-slave-7153 krb5kdc[31569](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:08:05 dist-test-slave-7153 krb5kdc[31569](info): set up 2 sockets
Sep 26 15:08:05 dist-test-slave-7153 krb5kdc[31569](info): commencing operation
krb5kdc: starting...
W20260926 15:08:06.249514 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.994s user 0.005s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:08:06 dist-test-slave-7153 krb5kdc[31569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:08:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:08:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:08:06.502386 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:42979
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:43027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:42979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:08:06.829802 31582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:08:06.830101 31582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:08:06.830224 31582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:08:06.837502 31582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:08:06.837625 31582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:08:06.837702 31582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:08:06.837777 31582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:08:06.837838 31582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:08:06.847829 31582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:43027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:42979
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:42979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:08:06.849189 31582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:08:06.850503 31582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:08:06.862303 31590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:06.862360 31588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:06.862360 31587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:08:06.863250 31582 server_base.cc:1034] running on GCE node
I20260926 15:08:06.864113 31582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:08:06.865636 31582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:08:06.866873 31582 hybrid_clock.cc:648] HybridClock initialized: now 1790435286866804 us; error 54 us; skew 500 ppm
I20260926 15:08:06.867275 31582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:08:06 dist-test-slave-7153 krb5kdc[31569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435286, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:08:06.870898 31582 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:08:06.872552 31582 webserver.cc:460] Webserver started at http://127.9.123.126:36187/ using document root <none> and password file <none>
I20260926 15:08:06.873378 31582 fs_manager.cc:359] Metadata directory not provided
I20260926 15:08:06.873509 31582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:08:06.873844 31582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:08:06.877133 31582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "230cfcbfd1724acd9094d89b0673545f"
format_stamp: "Formatted at 2026-09-26 15:08:06 on dist-test-slave-7153"
server_key: "23ff9495caabe096f651d6b6ce96af75"
server_key_iv: "d79ba23e7b1380f30be8bd90eabcafe9"
server_key_version: "encryptionkey@0"
I20260926 15:08:06.877992 31582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "230cfcbfd1724acd9094d89b0673545f"
format_stamp: "Formatted at 2026-09-26 15:08:06 on dist-test-slave-7153"
server_key: "23ff9495caabe096f651d6b6ce96af75"
server_key_iv: "d79ba23e7b1380f30be8bd90eabcafe9"
server_key_version: "encryptionkey@0"
I20260926 15:08:06.883474 31582 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 15:08:06.887118 31597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:08:06.888515 31582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 15:08:06.888687 31582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "230cfcbfd1724acd9094d89b0673545f"
format_stamp: "Formatted at 2026-09-26 15:08:06 on dist-test-slave-7153"
server_key: "23ff9495caabe096f651d6b6ce96af75"
server_key_iv: "d79ba23e7b1380f30be8bd90eabcafe9"
server_key_version: "encryptionkey@0"
I20260926 15:08:06.888890 31582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:08:06.899966 31582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:08:06.904153 31582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:08:06.904472 31582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:08:06.922003 31582 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:42979
I20260926 15:08:06.922015 31648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:42979 every 8 connection(s)
I20260926 15:08:06.923444 31582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:08:06.926298 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 31582
I20260926 15:08:06.926486 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:08:06.927106 9709 external_mini_cluster.cc:1428] Setting key 09d5bebfe081cabcdc7bfc9ce4bc855f
I20260926 15:08:06.927713 31649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:08:06.940035 31652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:06.928919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36109 (local address 127.9.123.126:42979)
0926 15:08:06.929660 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:06.929673 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:06.929712 (+ 39us) server_negotiation.cc:407] Connection header received
0926 15:08:06.931333 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:06.931354 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:06.931755 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:06.932138 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:06.932678 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:06.934050 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:06.935369 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:06.936030 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:06.936193 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:06.937637 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:06.937648 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:06.937663 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:08:06.938315 (+ 652us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:06.938399 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:08:06.938425 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:06.938568 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:06.938843 (+ 275us) server_negotiation.cc:299] Negotiation successful
0926 15:08:06.939024 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":322,"threads_started":1}
I20260926 15:08:06.940101 31649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 230cfcbfd1724acd9094d89b0673545f: Bootstrap starting.
I20260926 15:08:06.943702 31649 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 230cfcbfd1724acd9094d89b0673545f: Neither blocks nor log segments found. Creating new log.
I20260926 15:08:06.944169 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:08:06.944831 31649 log.cc:824] T 00000000000000000000000000000000 P 230cfcbfd1724acd9094d89b0673545f: Log is configured to *not* fsync() on all Append() calls
I20260926 15:08:06.947691 31649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 230cfcbfd1724acd9094d89b0673545f: No bootstrap required, opened a new log
I20260926 15:08:06.955754 31652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:06.948479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35829 (local address 127.9.123.126:42979)
0926 15:08:06.948667 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:06.948674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:06.949173 (+ 499us) server_negotiation.cc:407] Connection header received
0926 15:08:06.949407 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:06.949411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:06.949490 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:06.949593 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:06.950204 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:06.951038 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:06.952919 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:06.953507 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:06.953625 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:06.954925 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:06.954932 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:06.954935 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:08:06.954955 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:06.955007 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:08:06.955025 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:06.955128 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:06.955338 (+ 210us) server_negotiation.cc:299] Negotiation successful
0926 15:08:06.955395 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260926 15:08:06.957216 31649 raft_consensus.cc:348] T 00000000000000000000000000000000 P 230cfcbfd1724acd9094d89b0673545f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "230cfcbfd1724acd9094d89b0673545f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42979 } }
I20260926 15:08:06.957605 31649 raft_consensus.cc:374] T 00000000000000000000000000000000 P 230cfcbfd1724acd9094d89b0673545f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:08:06.957696 31649 raft_consensus.cc:729] T 00000000000000000000000000000000 P 230cfcbfd1724acd9094d89b0673545f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 230cfcbfd1724acd9094d89b0673545f, State: Initialized, Role: FOLLOWER
I20260926 15:08:06.958248 31649 consensus_queue.cc:260] T 00000000000000000000000000000000 P 230cfcbfd1724acd9094d89b0673545f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "230cfcbfd1724acd9094d89b0673545f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42979 } }
I20260926 15:08:06.958469 31649 raft_consensus.cc:388] T 00000000000000000000000000000000 P 230cfcbfd1724acd9094d89b0673545f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:08:06.958596 31649 raft_consensus.cc:482] T 00000000000000000000000000000000 P 230cfcbfd1724acd9094d89b0673545f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:08:06.958751 31649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 230cfcbfd1724acd9094d89b0673545f [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:08:06.960918 31649 raft_consensus.cc:504] T 00000000000000000000000000000000 P 230cfcbfd1724acd9094d89b0673545f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "230cfcbfd1724acd9094d89b0673545f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42979 } }
I20260926 15:08:06.961477 31649 leader_election.cc:302] T 00000000000000000000000000000000 P 230cfcbfd1724acd9094d89b0673545f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 230cfcbfd1724acd9094d89b0673545f; no voters:
I20260926 15:08:06.962045 31649 leader_election.cc:288] T 00000000000000000000000000000000 P 230cfcbfd1724acd9094d89b0673545f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:08:06.962198 31659 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 230cfcbfd1724acd9094d89b0673545f [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:08:06.963157 31659 raft_consensus.cc:686] T 00000000000000000000000000000000 P 230cfcbfd1724acd9094d89b0673545f [term 1 LEADER]: Becoming Leader. State: Replica: 230cfcbfd1724acd9094d89b0673545f, State: Running, Role: LEADER
I20260926 15:08:06.963802 31659 consensus_queue.cc:237] T 00000000000000000000000000000000 P 230cfcbfd1724acd9094d89b0673545f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "230cfcbfd1724acd9094d89b0673545f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42979 } }
I20260926 15:08:06.964227 31649 sys_catalog.cc:564] T 00000000000000000000000000000000 P 230cfcbfd1724acd9094d89b0673545f [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:08:06.969303 31661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 230cfcbfd1724acd9094d89b0673545f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 230cfcbfd1724acd9094d89b0673545f. Latest consensus state: current_term: 1 leader_uuid: "230cfcbfd1724acd9094d89b0673545f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "230cfcbfd1724acd9094d89b0673545f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42979 } } }
I20260926 15:08:06.969696 31661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 230cfcbfd1724acd9094d89b0673545f [sys.catalog]: This master's current role is: LEADER
I20260926 15:08:06.969725 31660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 230cfcbfd1724acd9094d89b0673545f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "230cfcbfd1724acd9094d89b0673545f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "230cfcbfd1724acd9094d89b0673545f" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42979 } } }
I20260926 15:08:06.970083 31660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 230cfcbfd1724acd9094d89b0673545f [sys.catalog]: This master's current role is: LEADER
I20260926 15:08:06.972355 31666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:08:06.977777 31666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:08:06.987290 31666 catalog_manager.cc:1347] Generated new cluster ID: 629e799e8c3d46cca0d9cc0e1e05da8a
I20260926 15:08:06.987440 31666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:08:07.002194 31666 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:08:07.003579 31666 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:08:07.010987 31666 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 230cfcbfd1724acd9094d89b0673545f: Generated new TSK 0
I20260926 15:08:07.011893 31666 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:08:07.018852 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 31582
W20260926 15:08:07.020309 31644 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36109
W20260926 15:08:07.021133 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 31582: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 15:08:07.021291 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 31582
W20260926 15:08:07.035002 31567 connection.cc:535] client connection to 127.9.123.126:42979 recv error: Network error: failed to read from TLS socket (remote: 127.9.123.126:42979): Connection reset by peer (error 104)
2026-09-26T15:08:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1797 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:08:07 dist-test-slave-7153 krb5kdc[31680](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:08:07 dist-test-slave-7153 krb5kdc[31680](info): set up 2 sockets
Sep 26 15:08:07 dist-test-slave-7153 krb5kdc[31680](info): commencing operation
krb5kdc: starting...
W20260926 15:08:08.053570 9709 mini_kdc.cc:121] Time spent starting KDC: real 1.000s user 0.006s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:08:08 dist-test-slave-7153 krb5kdc[31680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:08:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:08:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:08:08.303939 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:41661
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:38337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:41661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:08:08.639911 31693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:08:08.640234 31693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:08:08.640378 31693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:08:08.647702 31693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:08:08.647845 31693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:08:08.647925 31693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:08:08.648005 31693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:08:08.648063 31693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:08:08.658196 31693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:38337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:41661
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:41661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:08:08.659544 31693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:08:08.660882 31693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:08:08.672645 31699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:08.672622 31701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:08.672585 31698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:08:08.673623 31693 server_base.cc:1034] running on GCE node
I20260926 15:08:08.674424 31693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:08:08.675896 31693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:08:08.677191 31693 hybrid_clock.cc:648] HybridClock initialized: now 1790435288677122 us; error 55 us; skew 500 ppm
I20260926 15:08:08.677564 31693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:08:08 dist-test-slave-7153 krb5kdc[31680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435288, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:08:08.680763 31693 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:08:08.682392 31693 webserver.cc:460] Webserver started at http://127.9.123.126:40665/ using document root <none> and password file <none>
I20260926 15:08:08.683092 31693 fs_manager.cc:359] Metadata directory not provided
I20260926 15:08:08.683223 31693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:08:08.683519 31693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:08:08.686712 31693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "c8f049a8e33849b994826eced671839b"
format_stamp: "Formatted at 2026-09-26 15:08:08 on dist-test-slave-7153"
server_key: "94fb619bde21da398f936dc8e635cd98"
server_key_iv: "ee3b23e33fd4982077b3971853fe7111"
server_key_version: "encryptionkey@0"
I20260926 15:08:08.687530 31693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "c8f049a8e33849b994826eced671839b"
format_stamp: "Formatted at 2026-09-26 15:08:08 on dist-test-slave-7153"
server_key: "94fb619bde21da398f936dc8e635cd98"
server_key_iv: "ee3b23e33fd4982077b3971853fe7111"
server_key_version: "encryptionkey@0"
I20260926 15:08:08.693001 31693 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260926 15:08:08.696561 31708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:08:08.697969 31693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260926 15:08:08.698143 31693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "c8f049a8e33849b994826eced671839b"
format_stamp: "Formatted at 2026-09-26 15:08:08 on dist-test-slave-7153"
server_key: "94fb619bde21da398f936dc8e635cd98"
server_key_iv: "ee3b23e33fd4982077b3971853fe7111"
server_key_version: "encryptionkey@0"
I20260926 15:08:08.698338 31693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:08:08.704192 31693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:08:08.709388 31693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:08:08.709709 31693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:08:08.728201 31693 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:41661
I20260926 15:08:08.728253 31759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:41661 every 8 connection(s)
I20260926 15:08:08.729627 31693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:08:08.733572 31760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:08:08.738055 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 31693
I20260926 15:08:08.738257 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:08:08.738780 9709 external_mini_cluster.cc:1428] Setting key bed14bb1f40bf013a5b947e2cc1fe7b2
I20260926 15:08:08.745672 31760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8f049a8e33849b994826eced671839b: Bootstrap starting.
I20260926 15:08:08.747634 31763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:08.740756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39831 (local address 127.9.123.126:41661)
0926 15:08:08.741639 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:08.741654 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:08.741700 (+ 46us) server_negotiation.cc:407] Connection header received
0926 15:08:08.743910 (+ 2210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:08.743941 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:08.744359 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:08.744747 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:08.745978 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:08.745989 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:08.746004 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:08:08.746020 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:08.746090 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:08:08.746112 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:08.746303 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:08.746418 (+ 115us) server_negotiation.cc:299] Negotiation successful
0926 15:08:08.746627 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":421,"threads_started":1}
I20260926 15:08:08.749436 31760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8f049a8e33849b994826eced671839b: Neither blocks nor log segments found. Creating new log.
I20260926 15:08:08.750594 31760 log.cc:824] T 00000000000000000000000000000000 P c8f049a8e33849b994826eced671839b: Log is configured to *not* fsync() on all Append() calls
I20260926 15:08:08.751389 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:08:08.753849 31760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8f049a8e33849b994826eced671839b: No bootstrap required, opened a new log
I20260926 15:08:08.759043 31763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:08.756340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39069 (local address 127.9.123.126:41661)
0926 15:08:08.756530 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:08.756537 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:08.756963 (+ 426us) server_negotiation.cc:407] Connection header received
0926 15:08:08.757151 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:08.757156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:08.757223 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:08.757338 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:08.758252 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:08.758258 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:08.758262 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:08:08.758284 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:08.758335 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:08:08.758355 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:08.758483 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:08.758646 (+ 163us) server_negotiation.cc:299] Negotiation successful
0926 15:08:08.758717 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260926 15:08:08.762910 31760 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8f049a8e33849b994826eced671839b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f049a8e33849b994826eced671839b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 41661 } }
I20260926 15:08:08.763341 31760 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8f049a8e33849b994826eced671839b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:08:08.763453 31760 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8f049a8e33849b994826eced671839b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8f049a8e33849b994826eced671839b, State: Initialized, Role: FOLLOWER
I20260926 15:08:08.764081 31760 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8f049a8e33849b994826eced671839b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f049a8e33849b994826eced671839b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 41661 } }
I20260926 15:08:08.764314 31760 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8f049a8e33849b994826eced671839b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:08:08.764454 31760 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8f049a8e33849b994826eced671839b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:08:08.764603 31760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8f049a8e33849b994826eced671839b [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:08:08.767606 31760 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8f049a8e33849b994826eced671839b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f049a8e33849b994826eced671839b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 41661 } }
I20260926 15:08:08.768210 31760 leader_election.cc:302] T 00000000000000000000000000000000 P c8f049a8e33849b994826eced671839b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8f049a8e33849b994826eced671839b; no voters:
I20260926 15:08:08.768852 31760 leader_election.cc:288] T 00000000000000000000000000000000 P c8f049a8e33849b994826eced671839b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:08:08.768986 31770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8f049a8e33849b994826eced671839b [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:08:08.769670 31770 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8f049a8e33849b994826eced671839b [term 1 LEADER]: Becoming Leader. State: Replica: c8f049a8e33849b994826eced671839b, State: Running, Role: LEADER
I20260926 15:08:08.770358 31770 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8f049a8e33849b994826eced671839b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f049a8e33849b994826eced671839b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 41661 } }
I20260926 15:08:08.771147 31760 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8f049a8e33849b994826eced671839b [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:08:08.773507 31772 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8f049a8e33849b994826eced671839b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8f049a8e33849b994826eced671839b. Latest consensus state: current_term: 1 leader_uuid: "c8f049a8e33849b994826eced671839b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f049a8e33849b994826eced671839b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 41661 } } }
I20260926 15:08:08.773589 31771 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8f049a8e33849b994826eced671839b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8f049a8e33849b994826eced671839b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f049a8e33849b994826eced671839b" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 41661 } } }
I20260926 15:08:08.773978 31772 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8f049a8e33849b994826eced671839b [sys.catalog]: This master's current role is: LEADER
I20260926 15:08:08.773986 31771 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8f049a8e33849b994826eced671839b [sys.catalog]: This master's current role is: LEADER
I20260926 15:08:08.775657 31778 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:08:08.781075 31778 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:08:08.791280 31778 catalog_manager.cc:1347] Generated new cluster ID: 10b0ff172c22488ea6a31ffd35b80384
I20260926 15:08:08.791400 31778 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:08:08.827373 31778 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:08:08.827829 31778 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:08:08.839304 31778 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8f049a8e33849b994826eced671839b: Generated new TSK 0
I20260926 15:08:08.840159 31778 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:08:08.850589 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 31693
W20260926 15:08:08.851743 31755 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39831
W20260926 15:08:08.852504 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 31693: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 15:08:08.852663 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 31693
2026-09-26T15:08:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1831 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:08:08 dist-test-slave-7153 krb5kdc[31791](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:08:08 dist-test-slave-7153 krb5kdc[31791](info): set up 2 sockets
Sep 26 15:08:08 dist-test-slave-7153 krb5kdc[31791](info): commencing operation
krb5kdc: starting...
W20260926 15:08:09.877430 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:08:10 dist-test-slave-7153 krb5kdc[31791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:08:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:08:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:08:10.125694 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:40025
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:42637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:40025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:08:10.457775 31804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:08:10.458082 31804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:08:10.458176 31804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:08:10.465565 31804 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:08:10.465701 31804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:08:10.465749 31804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:08:10.465809 31804 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:08:10.465852 31804 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:08:10.475919 31804 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:42637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:40025
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:40025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:08:10.477257 31804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:08:10.478542 31804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:08:10.490089 31812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:10.490061 31810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:10.490134 31809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:08:10.490685 31804 server_base.cc:1034] running on GCE node
I20260926 15:08:10.491438 31804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:08:10.492828 31804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:08:10.494064 31804 hybrid_clock.cc:648] HybridClock initialized: now 1790435290494007 us; error 45 us; skew 500 ppm
I20260926 15:08:10.494451 31804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:08:10 dist-test-slave-7153 krb5kdc[31791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435290, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:08:10.497604 31804 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:08:10.499092 31804 webserver.cc:460] Webserver started at http://127.9.123.126:43689/ using document root <none> and password file <none>
I20260926 15:08:10.499814 31804 fs_manager.cc:359] Metadata directory not provided
I20260926 15:08:10.499936 31804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:08:10.500257 31804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:08:10.503310 31804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "43c78bde9bca4d00a152a8af8477a456"
format_stamp: "Formatted at 2026-09-26 15:08:10 on dist-test-slave-7153"
server_key: "06b16fb58665a8a0064b0d89d7cf864e"
server_key_iv: "e9f1d83f623ec5379c2fa603bbcdb81a"
server_key_version: "encryptionkey@0"
I20260926 15:08:10.504082 31804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "43c78bde9bca4d00a152a8af8477a456"
format_stamp: "Formatted at 2026-09-26 15:08:10 on dist-test-slave-7153"
server_key: "06b16fb58665a8a0064b0d89d7cf864e"
server_key_iv: "e9f1d83f623ec5379c2fa603bbcdb81a"
server_key_version: "encryptionkey@0"
I20260926 15:08:10.509320 31804 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260926 15:08:10.512898 31819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:08:10.514278 31804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:08:10.514442 31804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "43c78bde9bca4d00a152a8af8477a456"
format_stamp: "Formatted at 2026-09-26 15:08:10 on dist-test-slave-7153"
server_key: "06b16fb58665a8a0064b0d89d7cf864e"
server_key_iv: "e9f1d83f623ec5379c2fa603bbcdb81a"
server_key_version: "encryptionkey@0"
I20260926 15:08:10.514629 31804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:08:10.530205 31804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:08:10.534312 31804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:08:10.534576 31804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:08:10.551467 31804 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:40025
I20260926 15:08:10.551477 31870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:40025 every 8 connection(s)
I20260926 15:08:10.552848 31804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:08:10.556793 31871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:08:10.559439 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 31804
I20260926 15:08:10.559616 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:08:10.560130 9709 external_mini_cluster.cc:1428] Setting key 2c9b459fac4f828a2c6127a3fde5ac64
I20260926 15:08:10.569059 31871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43c78bde9bca4d00a152a8af8477a456: Bootstrap starting.
I20260926 15:08:10.572245 31871 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43c78bde9bca4d00a152a8af8477a456: Neither blocks nor log segments found. Creating new log.
I20260926 15:08:10.573331 31871 log.cc:824] T 00000000000000000000000000000000 P 43c78bde9bca4d00a152a8af8477a456: Log is configured to *not* fsync() on all Append() calls
Sep 26 15:08:10 dist-test-slave-7153 krb5kdc[31791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:08:10.576476 31871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43c78bde9bca4d00a152a8af8477a456: No bootstrap required, opened a new log
I20260926 15:08:10.584563 31874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:10.562376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48073 (local address 127.9.123.126:40025)
0926 15:08:10.563247 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:10.563261 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:10.563303 (+ 42us) server_negotiation.cc:407] Connection header received
0926 15:08:10.565077 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:10.565138 (+ 61us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:10.565606 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:10.566029 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:10.567471 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:10.568810 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:10.570175 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:10.570896 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:10.571067 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:10.575338 (+ 4271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:10.575346 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:10.575361 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:10.575396 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:10.578467 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:10.579414 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:10.579421 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:10.579428 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:10.579684 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:10.580147 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:10.580152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:10.580154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:10.580805 (+ 651us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:10.582144 (+ 1339us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:10.582468 (+ 324us) server_negotiation.cc:299] Negotiation successful
0926 15:08:10.583005 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":386,"threads_started":1}
I20260926 15:08:10.586408 31871 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43c78bde9bca4d00a152a8af8477a456 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c78bde9bca4d00a152a8af8477a456" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 40025 } }
I20260926 15:08:10.586792 31871 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43c78bde9bca4d00a152a8af8477a456 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:08:10.586894 31871 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43c78bde9bca4d00a152a8af8477a456 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43c78bde9bca4d00a152a8af8477a456, State: Initialized, Role: FOLLOWER
I20260926 15:08:10.587513 31871 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43c78bde9bca4d00a152a8af8477a456 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c78bde9bca4d00a152a8af8477a456" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 40025 } }
I20260926 15:08:10.587857 31871 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43c78bde9bca4d00a152a8af8477a456 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:08:10.587989 31871 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43c78bde9bca4d00a152a8af8477a456 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:08:10.588146 31871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43c78bde9bca4d00a152a8af8477a456 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:08:10.589001 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:08:10.591388 31871 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43c78bde9bca4d00a152a8af8477a456 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c78bde9bca4d00a152a8af8477a456" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 40025 } }
I20260926 15:08:10.592046 31871 leader_election.cc:302] T 00000000000000000000000000000000 P 43c78bde9bca4d00a152a8af8477a456 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43c78bde9bca4d00a152a8af8477a456; no voters:
I20260926 15:08:10.592756 31871 leader_election.cc:288] T 00000000000000000000000000000000 P 43c78bde9bca4d00a152a8af8477a456 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:08:10.592878 31877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43c78bde9bca4d00a152a8af8477a456 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:08:10.594081 31877 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43c78bde9bca4d00a152a8af8477a456 [term 1 LEADER]: Becoming Leader. State: Replica: 43c78bde9bca4d00a152a8af8477a456, State: Running, Role: LEADER
I20260926 15:08:10.594744 31877 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43c78bde9bca4d00a152a8af8477a456 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c78bde9bca4d00a152a8af8477a456" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 40025 } }
I20260926 15:08:10.595336 31871 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43c78bde9bca4d00a152a8af8477a456 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:08:10.600329 31881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43c78bde9bca4d00a152a8af8477a456 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43c78bde9bca4d00a152a8af8477a456" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c78bde9bca4d00a152a8af8477a456" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 40025 } } }
I20260926 15:08:10.600726 31881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43c78bde9bca4d00a152a8af8477a456 [sys.catalog]: This master's current role is: LEADER
I20260926 15:08:10.601312 31882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43c78bde9bca4d00a152a8af8477a456 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43c78bde9bca4d00a152a8af8477a456. Latest consensus state: current_term: 1 leader_uuid: "43c78bde9bca4d00a152a8af8477a456" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c78bde9bca4d00a152a8af8477a456" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 40025 } } }
I20260926 15:08:10.601620 31882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43c78bde9bca4d00a152a8af8477a456 [sys.catalog]: This master's current role is: LEADER
I20260926 15:08:10.606626 31889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:08:10.612730 31889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:08:10.613940 31874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:10.594573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42747 (local address 127.9.123.126:40025)
0926 15:08:10.594836 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:10.594842 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:10.595409 (+ 567us) server_negotiation.cc:407] Connection header received
0926 15:08:10.595588 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:10.595593 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:10.595673 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:10.595776 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:10.597834 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:10.599027 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:10.600999 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:10.601802 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:10.601902 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:10.605339 (+ 3437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:10.605347 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:10.605352 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:10.605395 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:10.609392 (+ 3997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:10.610417 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:10.610424 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:10.610428 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:10.610602 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:10.611098 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:10.611103 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:10.611107 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:10.611598 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:10.613078 (+ 1480us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:10.613259 (+ 181us) server_negotiation.cc:299] Negotiation successful
0926 15:08:10.613524 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260926 15:08:10.623857 31889 catalog_manager.cc:1347] Generated new cluster ID: c274e0ab878047cab592b482e0c0abf9
I20260926 15:08:10.623991 31889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:08:10.624163 31898 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 15:08:10.637677 31889 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:08:10.639533 31889 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:08:10.652664 31889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43c78bde9bca4d00a152a8af8477a456: Generated new TSK 0
I20260926 15:08:10.653811 31889 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:08:10.660207 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 31804
I20260926 15:08:10.661388 31866 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:08:10.750552 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 31804
2026-09-26T15:08:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1897 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:08:10 dist-test-slave-7153 krb5kdc[31903](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:08:10 dist-test-slave-7153 krb5kdc[31903](info): set up 2 sockets
Sep 26 15:08:10 dist-test-slave-7153 krb5kdc[31903](info): commencing operation
krb5kdc: starting...
W20260926 15:08:11.773495 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.991s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:08:11 dist-test-slave-7153 krb5kdc[31903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:08:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:08:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:08:12.024611 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:34573
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:34561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:34573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:08:12.354117 31916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:08:12.354427 31916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:08:12.354533 31916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:08:12.362509 31916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:08:12.362664 31916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:08:12.362713 31916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:08:12.362779 31916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:08:12.362830 31916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:08:12.375046 31916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:34561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:34573
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:34573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:08:12.376421 31916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:08:12.377946 31916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:08:12.391479 31922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:12.391460 31924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:12.391322 31921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:08:12.392349 31916 server_base.cc:1034] running on GCE node
I20260926 15:08:12.393244 31916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:08:12.395069 31916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:08:12.396318 31916 hybrid_clock.cc:648] HybridClock initialized: now 1790435292396256 us; error 55 us; skew 500 ppm
I20260926 15:08:12.396853 31916 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:08:12 dist-test-slave-7153 krb5kdc[31903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435292, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:08:12.400947 31916 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:08:12.402621 31916 webserver.cc:460] Webserver started at http://127.9.123.126:46679/ using document root <none> and password file <none>
I20260926 15:08:12.403363 31916 fs_manager.cc:359] Metadata directory not provided
I20260926 15:08:12.403496 31916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:08:12.403811 31916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:08:12.407027 31916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "97869f5b8cf147c6aa41787cf824c531"
format_stamp: "Formatted at 2026-09-26 15:08:12 on dist-test-slave-7153"
server_key: "36dd488e6afb64999ae6cd0f51da2670"
server_key_iv: "c5185887b4cc91f3768dc32fc60864a3"
server_key_version: "encryptionkey@0"
I20260926 15:08:12.407912 31916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "97869f5b8cf147c6aa41787cf824c531"
format_stamp: "Formatted at 2026-09-26 15:08:12 on dist-test-slave-7153"
server_key: "36dd488e6afb64999ae6cd0f51da2670"
server_key_iv: "c5185887b4cc91f3768dc32fc60864a3"
server_key_version: "encryptionkey@0"
I20260926 15:08:12.413883 31916 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260926 15:08:12.418412 31931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:08:12.420027 31916 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260926 15:08:12.420238 31916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "97869f5b8cf147c6aa41787cf824c531"
format_stamp: "Formatted at 2026-09-26 15:08:12 on dist-test-slave-7153"
server_key: "36dd488e6afb64999ae6cd0f51da2670"
server_key_iv: "c5185887b4cc91f3768dc32fc60864a3"
server_key_version: "encryptionkey@0"
I20260926 15:08:12.420454 31916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:08:12.457211 31916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:08:12.461700 31916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:08:12.461980 31916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:08:12.479620 31916 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:34573
I20260926 15:08:12.479707 31982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:34573 every 8 connection(s)
I20260926 15:08:12.481017 31916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:08:12.485003 31983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:08:12.490165 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 31916
I20260926 15:08:12.490343 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:08:12.490788 9709 external_mini_cluster.cc:1428] Setting key 1cf762a440d14eb3b0cce7257bf00c5a
I20260926 15:08:12.495914 31983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531: Bootstrap starting.
I20260926 15:08:12.499547 31983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531: Neither blocks nor log segments found. Creating new log.
I20260926 15:08:12.500710 31983 log.cc:824] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531: Log is configured to *not* fsync() on all Append() calls
I20260926 15:08:12.504030 31983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531: No bootstrap required, opened a new log
I20260926 15:08:12.504606 31986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:12.492644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42875 (local address 127.9.123.126:34573)
0926 15:08:12.493541 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:12.493557 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:12.493601 (+ 44us) server_negotiation.cc:407] Connection header received
0926 15:08:12.495331 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:12.495355 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:12.495761 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:12.496093 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:12.496674 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:12.498031 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:12.499813 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:12.500563 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:12.500722 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:12.502162 (+ 1440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:12.502172 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:12.502191 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:08:12.502740 (+ 549us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:12.502815 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:08:12.502843 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:12.502936 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:12.503231 (+ 295us) server_negotiation.cc:299] Negotiation successful
0926 15:08:12.503515 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":411,"threads_started":1}
I20260926 15:08:12.508844 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:08:12.512458 31983 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97869f5b8cf147c6aa41787cf824c531" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 34573 } }
I20260926 15:08:12.512858 31983 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:08:12.512981 31983 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97869f5b8cf147c6aa41787cf824c531, State: Initialized, Role: FOLLOWER
I20260926 15:08:12.513958 31983 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97869f5b8cf147c6aa41787cf824c531" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 34573 } }
I20260926 15:08:12.514300 31993 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260926 15:08:12.514537 31993 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97869f5b8cf147c6aa41787cf824c531" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 34573 } }
I20260926 15:08:12.514257 31983 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:08:12.515259 31983 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:08:12.515224 31993 leader_election.cc:302] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97869f5b8cf147c6aa41787cf824c531; no voters:
I20260926 15:08:12.515373 31983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:08:12.516060 31993 leader_election.cc:288] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260926 15:08:12.518324 31983 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97869f5b8cf147c6aa41787cf824c531" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 34573 } }
I20260926 15:08:12.518558 31983 leader_election.cc:302] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97869f5b8cf147c6aa41787cf824c531; no voters:
I20260926 15:08:12.518738 31983 leader_election.cc:288] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:08:12.518916 31993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:08:12.519621 31993 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531 [term 1 LEADER]: Becoming Leader. State: Replica: 97869f5b8cf147c6aa41787cf824c531, State: Running, Role: LEADER
I20260926 15:08:12.520051 31986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:12.513032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35169 (local address 127.9.123.126:34573)
0926 15:08:12.513261 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:12.513268 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:12.513704 (+ 436us) server_negotiation.cc:407] Connection header received
0926 15:08:12.513925 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:12.513929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:12.514013 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:12.514101 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:12.514688 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:12.515767 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:12.517313 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:12.517983 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:12.518109 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:12.519177 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:12.519187 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:12.519192 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:08:12.519220 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:12.519284 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:08:12.519312 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:12.519382 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:12.519645 (+ 263us) server_negotiation.cc:299] Negotiation successful
0926 15:08:12.519749 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260926 15:08:12.520325 31993 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97869f5b8cf147c6aa41787cf824c531" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 34573 } }
I20260926 15:08:12.521006 31983 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:08:12.522226 31994 raft_consensus.cc:2741] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531 [term 1 LEADER]: Leader pre-election decision vote started in defunct term 0: won
I20260926 15:08:12.524696 31996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97869f5b8cf147c6aa41787cf824c531. Latest consensus state: current_term: 1 leader_uuid: "97869f5b8cf147c6aa41787cf824c531" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97869f5b8cf147c6aa41787cf824c531" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 34573 } } }
I20260926 15:08:12.524780 31995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97869f5b8cf147c6aa41787cf824c531" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97869f5b8cf147c6aa41787cf824c531" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 34573 } } }
I20260926 15:08:12.525205 31995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531 [sys.catalog]: This master's current role is: LEADER
I20260926 15:08:12.525308 31996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531 [sys.catalog]: This master's current role is: LEADER
I20260926 15:08:12.527029 32000 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:08:12.535737 32000 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:08:12.545730 32000 catalog_manager.cc:1347] Generated new cluster ID: 5125b41cace34dcb9001573dc32f632c
I20260926 15:08:12.545849 32000 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:08:12.562705 32000 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:08:12.564078 32000 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:08:12.580749 32000 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97869f5b8cf147c6aa41787cf824c531: Generated new TSK 0
I20260926 15:08:12.581789 32000 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:08:12.599606 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 31916
W20260926 15:08:12.600801 31978 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42875
W20260926 15:08:12.601570 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 31916: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 15:08:12.601707 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 31916
2026-09-26T15:08:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1851 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:08:12 dist-test-slave-7153 krb5kdc[32015](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:08:12 dist-test-slave-7153 krb5kdc[32015](info): set up 2 sockets
Sep 26 15:08:12 dist-test-slave-7153 krb5kdc[32015](info): commencing operation
krb5kdc: starting...
W20260926 15:08:13.617452 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.983s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:08:13 dist-test-slave-7153 krb5kdc[32015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:08:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:08:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:08:13.880187 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:45891
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:33827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:45891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:08:14.209134 32028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:08:14.209467 32028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:08:14.209610 32028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:08:14.216933 32028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:08:14.217077 32028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:08:14.217186 32028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:08:14.217267 32028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:08:14.217325 32028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:08:14.227449 32028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:33827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:45891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:45891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:08:14.228884 32028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:08:14.230274 32028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:08:14.242559 32033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:14.242599 32036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:14.242518 32034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:08:14.243758 32028 server_base.cc:1034] running on GCE node
I20260926 15:08:14.244663 32028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:08:14.246201 32028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:08:14.247460 32028 hybrid_clock.cc:648] HybridClock initialized: now 1790435294247396 us; error 47 us; skew 500 ppm
I20260926 15:08:14.247844 32028 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:08:14 dist-test-slave-7153 krb5kdc[32015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435294, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:08:14.255899 32028 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:08:14.257905 32028 webserver.cc:460] Webserver started at http://127.9.123.126:39779/ using document root <none> and password file <none>
I20260926 15:08:14.258888 32028 fs_manager.cc:359] Metadata directory not provided
I20260926 15:08:14.259048 32028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:08:14.259421 32028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:08:14.262773 32028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "d3115bd8d7884765911acf3c9bc9383e"
format_stamp: "Formatted at 2026-09-26 15:08:14 on dist-test-slave-7153"
server_key: "92eb46ddbb8e1b09bb488a20e64f45ae"
server_key_iv: "78aa585dcb2ed51eaa2401a1be7c2a2e"
server_key_version: "encryptionkey@0"
I20260926 15:08:14.263579 32028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "d3115bd8d7884765911acf3c9bc9383e"
format_stamp: "Formatted at 2026-09-26 15:08:14 on dist-test-slave-7153"
server_key: "92eb46ddbb8e1b09bb488a20e64f45ae"
server_key_iv: "78aa585dcb2ed51eaa2401a1be7c2a2e"
server_key_version: "encryptionkey@0"
I20260926 15:08:14.269088 32028 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260926 15:08:14.272915 32043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:08:14.274482 32028 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:08:14.274675 32028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "d3115bd8d7884765911acf3c9bc9383e"
format_stamp: "Formatted at 2026-09-26 15:08:14 on dist-test-slave-7153"
server_key: "92eb46ddbb8e1b09bb488a20e64f45ae"
server_key_iv: "78aa585dcb2ed51eaa2401a1be7c2a2e"
server_key_version: "encryptionkey@0"
I20260926 15:08:14.274875 32028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:08:14.280848 32028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:08:14.285199 32028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:08:14.285466 32028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:08:14.302976 32028 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:45891
I20260926 15:08:14.303009 32094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:45891 every 8 connection(s)
I20260926 15:08:14.304369 32028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:08:14.304665 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 32028
I20260926 15:08:14.304816 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:08:14.305282 9709 external_mini_cluster.cc:1428] Setting key b8c16cf791a431239162a00acc656f84
I20260926 15:08:14.309288 32095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:08:14.314352 32098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:14.307258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36229 (local address 127.9.123.126:45891)
0926 15:08:14.308082 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:14.308099 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:14.308150 (+ 51us) server_negotiation.cc:407] Connection header received
0926 15:08:14.310433 (+ 2283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:14.310465 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:14.310933 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:14.311392 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:14.312364 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:14.312376 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:14.312393 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:08:14.312409 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:14.312503 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:08:14.312532 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:14.312756 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:14.312891 (+ 135us) server_negotiation.cc:299] Negotiation successful
0926 15:08:14.313090 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":331,"threads_started":1}
I20260926 15:08:14.318645 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:08:14.322311 32095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3115bd8d7884765911acf3c9bc9383e: Bootstrap starting.
I20260926 15:08:14.325292 32098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:14.322799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37677 (local address 127.9.123.126:45891)
0926 15:08:14.322952 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:14.322956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:14.323470 (+ 514us) server_negotiation.cc:407] Connection header received
0926 15:08:14.323658 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:14.323663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:14.323714 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:14.323794 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:14.324661 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:14.324666 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:14.324671 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0926 15:08:14.324689 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:14.324727 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0926 15:08:14.324742 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:14.324837 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:14.324938 (+ 101us) server_negotiation.cc:299] Negotiation successful
0926 15:08:14.324996 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260926 15:08:14.325870 32095 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3115bd8d7884765911acf3c9bc9383e: Neither blocks nor log segments found. Creating new log.
I20260926 15:08:14.327005 32095 log.cc:824] T 00000000000000000000000000000000 P d3115bd8d7884765911acf3c9bc9383e: Log is configured to *not* fsync() on all Append() calls
I20260926 15:08:14.329875 32095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3115bd8d7884765911acf3c9bc9383e: No bootstrap required, opened a new log
I20260926 15:08:14.337324 32095 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3115bd8d7884765911acf3c9bc9383e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3115bd8d7884765911acf3c9bc9383e" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45891 } }
I20260926 15:08:14.337826 32095 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3115bd8d7884765911acf3c9bc9383e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260926 15:08:14.337939 32095 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3115bd8d7884765911acf3c9bc9383e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3115bd8d7884765911acf3c9bc9383e, State: Initialized, Role: FOLLOWER
I20260926 15:08:14.338549 32095 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3115bd8d7884765911acf3c9bc9383e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3115bd8d7884765911acf3c9bc9383e" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45891 } }
I20260926 15:08:14.338759 32095 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3115bd8d7884765911acf3c9bc9383e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260926 15:08:14.338887 32095 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3115bd8d7884765911acf3c9bc9383e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260926 15:08:14.339048 32095 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3115bd8d7884765911acf3c9bc9383e [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:08:14.341230 32095 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3115bd8d7884765911acf3c9bc9383e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3115bd8d7884765911acf3c9bc9383e" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45891 } }
I20260926 15:08:14.341724 32095 leader_election.cc:302] T 00000000000000000000000000000000 P d3115bd8d7884765911acf3c9bc9383e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3115bd8d7884765911acf3c9bc9383e; no voters:
I20260926 15:08:14.342314 32095 leader_election.cc:288] T 00000000000000000000000000000000 P d3115bd8d7884765911acf3c9bc9383e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260926 15:08:14.342489 32105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3115bd8d7884765911acf3c9bc9383e [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:08:14.343264 32105 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3115bd8d7884765911acf3c9bc9383e [term 1 LEADER]: Becoming Leader. State: Replica: d3115bd8d7884765911acf3c9bc9383e, State: Running, Role: LEADER
I20260926 15:08:14.343844 32105 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3115bd8d7884765911acf3c9bc9383e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3115bd8d7884765911acf3c9bc9383e" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45891 } }
I20260926 15:08:14.344376 32095 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3115bd8d7884765911acf3c9bc9383e [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:08:14.346707 32106 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3115bd8d7884765911acf3c9bc9383e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3115bd8d7884765911acf3c9bc9383e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3115bd8d7884765911acf3c9bc9383e" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45891 } } }
I20260926 15:08:14.347244 32106 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3115bd8d7884765911acf3c9bc9383e [sys.catalog]: This master's current role is: LEADER
I20260926 15:08:14.347373 32107 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3115bd8d7884765911acf3c9bc9383e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3115bd8d7884765911acf3c9bc9383e. Latest consensus state: current_term: 1 leader_uuid: "d3115bd8d7884765911acf3c9bc9383e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3115bd8d7884765911acf3c9bc9383e" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 45891 } } }
I20260926 15:08:14.347714 32107 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3115bd8d7884765911acf3c9bc9383e [sys.catalog]: This master's current role is: LEADER
I20260926 15:08:14.348349 32113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:08:14.353051 32113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:08:14.362301 32113 catalog_manager.cc:1347] Generated new cluster ID: 88c104e6f37a462e8b08f3f011e3df95
I20260926 15:08:14.362424 32113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:08:14.380353 32113 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:08:14.380909 32113 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:08:14.388145 32113 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3115bd8d7884765911acf3c9bc9383e: Generated new TSK 0
I20260926 15:08:14.389257 32113 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:08:14.405449 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 32028
W20260926 15:08:14.406937 32090 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36229
W20260926 15:08:14.407927 9709 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 32028: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260926 15:08:14.408124 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 32028
2026-09-26T15:08:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1806 ms)
[----------] 18 tests from AuthTokenIssuingTest (33416 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:08:14 dist-test-slave-7153 krb5kdc[32126](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:08:14 dist-test-slave-7153 krb5kdc[32126](info): set up 2 sockets
Sep 26 15:08:14 dist-test-slave-7153 krb5kdc[32126](info): commencing operation
krb5kdc: starting...
W20260926 15:08:15.433638 9709 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:08:15 dist-test-slave-7153 krb5kdc[32126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:08:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:08:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:08:15.693956 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:42713
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:34837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:42713,127.9.123.125:36585,127.9.123.124:35867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:08:16.020604 32139 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 15:08:16.020943 32139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:08:16.021049 32139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:08:16.021185 32139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:08:16.028630 32139 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:08:16.028738 32139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:08:16.028800 32139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:08:16.028872 32139 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:08:16.028952 32139 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:08:16.039443 32139 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:34837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:42713,127.9.123.125:36585,127.9.123.124:35867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:42713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:08:16.040807 32139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:08:16.042203 32139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:08:16.054068 32144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:16.054050 32145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:16.054090 32147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:08:16.054807 32139 server_base.cc:1034] running on GCE node
I20260926 15:08:16.055626 32139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:08:16.057126 32139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:08:16.058365 32139 hybrid_clock.cc:648] HybridClock initialized: now 1790435296058302 us; error 43 us; skew 500 ppm
I20260926 15:08:16.058709 32139 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:08:16 dist-test-slave-7153 krb5kdc[32126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435296, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:08:16.061892 32139 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:08:16.063407 32139 webserver.cc:460] Webserver started at http://127.9.123.126:34855/ using document root <none> and password file <none>
I20260926 15:08:16.064098 32139 fs_manager.cc:359] Metadata directory not provided
I20260926 15:08:16.064199 32139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:08:16.064496 32139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:08:16.067534 32139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4"
format_stamp: "Formatted at 2026-09-26 15:08:16 on dist-test-slave-7153"
server_key: "d6564bb5eac6b3e63d4edc88017f30b0"
server_key_iv: "a58c792063cd75fbfb3d28d6f7575e56"
server_key_version: "encryptionkey@0"
I20260926 15:08:16.068284 32139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4"
format_stamp: "Formatted at 2026-09-26 15:08:16 on dist-test-slave-7153"
server_key: "d6564bb5eac6b3e63d4edc88017f30b0"
server_key_iv: "a58c792063cd75fbfb3d28d6f7575e56"
server_key_version: "encryptionkey@0"
I20260926 15:08:16.073532 32139 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260926 15:08:16.077049 32154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:08:16.078567 32139 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:08:16.078749 32139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4"
format_stamp: "Formatted at 2026-09-26 15:08:16 on dist-test-slave-7153"
server_key: "d6564bb5eac6b3e63d4edc88017f30b0"
server_key_iv: "a58c792063cd75fbfb3d28d6f7575e56"
server_key_version: "encryptionkey@0"
I20260926 15:08:16.078965 32139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:08:16.112159 32139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:08:16.116817 32139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:08:16.117144 32139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:08:16.134933 32139 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:42713
I20260926 15:08:16.134940 32205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:42713 every 8 connection(s)
I20260926 15:08:16.136303 32139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:08:16.138561 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 32139
I20260926 15:08:16.138767 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:08:16.139575 9709 external_mini_cluster.cc:1428] Setting key fc7c619fc0ec99cc1764f6a22b551a9a
I20260926 15:08:16.140930 32206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:08:16.149736 32206 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42713 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:08:16 dist-test-slave-7153 krb5kdc[32126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435296, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.123.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.125@KRBTEST.COM" created.
I20260926 15:08:16.174849 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.153133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:42713 (local address 127.0.0.1:40790)
0926 15:08:16.153988 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.154046 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0926 15:08:16.156492 (+ 2446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:08:16.157519 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:08:16.157527 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:08:16.158278 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:08:16.158757 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:16.158785 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:16.160402 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:16.160408 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:16.162302 (+ 1894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:16.162318 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:16.163468 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:16.163476 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:16.163686 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:16.164418 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:08:16.164452 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:08:16.167143 (+ 2691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:08:16.170693 (+ 3550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:16.170704 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:16.170715 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:16.171113 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:16.171612 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:16.171616 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:16.171618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:16.171798 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:16.172767 (+ 969us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:08:16.172774 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:08:16.172979 (+ 205us) client_negotiation.cc:769] Sending connection context
0926 15:08:16.173241 (+ 262us) client_negotiation.cc:240] Negotiation successful
0926 15:08:16.173713 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":311,"threads_started":1}
I20260926 15:08:16.174881 32210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.154278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40790 (local address 127.9.123.126:42713)
0926 15:08:16.154983 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:16.154992 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:16.155038 (+ 46us) server_negotiation.cc:407] Connection header received
0926 15:08:16.156952 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:16.156960 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:16.157265 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:16.157431 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:16.159032 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:16.160235 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:16.162503 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:16.163198 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:16.163340 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:16.167390 (+ 4050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:16.167399 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:16.167413 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:16.167446 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:16.170530 (+ 3084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:16.171275 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:16.171283 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:16.171286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:16.171482 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:16.171906 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:16.171910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:16.171914 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:16.172593 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:16.172790 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:16.173450 (+ 660us) server_negotiation.cc:299] Negotiation successful
0926 15:08:16.173904 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":148,"server-negotiator.queue_time_us":544,"thread_start_us":302,"threads_started":1}
I20260926 15:08:16.178072 32206 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:08:16.179234 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.178780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:36585 (local address 127.0.0.1:44846)
0926 15:08:16.178985 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.179106 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:36585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260926 15:08:16.179512 32157 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.123.125:36585: connect: Connection refused (error 111)
I20260926 15:08:16.181326 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.180819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:36585 (local address 127.0.0.1:44854)
0926 15:08:16.180994 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.181127 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:36585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 15:08:16.181556 32206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.125:36585: Network error: Client connection negotiation failed: client connection to 127.9.123.125:36585: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.125.keytab.
Entry for principal kudu/127.9.123.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.125.keytab.
I20260926 15:08:16.204797 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.125:36585
--webserver_interface=127.9.123.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.125.keytab
--principal=kudu/127.9.123.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:34837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:42713,127.9.123.125:36585,127.9.123.124:35867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:08:16.230929 32206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } attempt: 1
I20260926 15:08:16.232191 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.231668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:36585 (local address 127.0.0.1:44860)
0926 15:08:16.231888 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.232018 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:36585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260926 15:08:16.233510 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.232944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:36585 (local address 127.0.0.1:44870)
0926 15:08:16.233134 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.233231 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:36585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260926 15:08:16.233749 32206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.125:36585: Network error: Client connection negotiation failed: client connection to 127.9.123.125:36585: connect: Connection refused (error 111)
I20260926 15:08:16.302037 32206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } attempt: 2
I20260926 15:08:16.303435 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.302866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:36585 (local address 127.0.0.1:44880)
0926 15:08:16.303104 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.303235 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:36585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260926 15:08:16.304662 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.304138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:36585 (local address 127.0.0.1:44892)
0926 15:08:16.304291 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.304411 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:36585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260926 15:08:16.304965 32206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.125:36585: Network error: Client connection negotiation failed: client connection to 127.9.123.125:36585: connect: Connection refused (error 111)
I20260926 15:08:16.396265 32206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } attempt: 3
I20260926 15:08:16.397579 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.397013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:36585 (local address 127.0.0.1:44908)
0926 15:08:16.397270 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.397384 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:36585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260926 15:08:16.398754 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.398305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:36585 (local address 127.0.0.1:44916)
0926 15:08:16.398464 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.398570 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:36585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260926 15:08:16.398967 32206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.125:36585: Network error: Client connection negotiation failed: client connection to 127.9.123.125:36585: connect: Connection refused (error 111)
W20260926 15:08:16.535601 32213 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 15:08:16.535946 32213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:08:16.536043 32213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:08:16.536155 32213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260926 15:08:16.542217 32206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } attempt: 4
I20260926 15:08:16.543479 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.542881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:36585 (local address 127.0.0.1:44918)
0926 15:08:16.543209 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.543317 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:36585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
W20260926 15:08:16.544128 32213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:08:16.544217 32213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:08:16.544370 32213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:08:16.544452 32213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:08:16.544538 32213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:08:16.544564 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.544148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:36585 (local address 127.0.0.1:44920)
0926 15:08:16.544304 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.544410 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:36585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 15:08:16.544759 32206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.125:36585: Network error: Client connection negotiation failed: client connection to 127.9.123.125:36585: connect: Connection refused (error 111)
I20260926 15:08:16.555349 32213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:34837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:42713,127.9.123.125:36585,127.9.123.124:35867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.125.keytab
--principal=kudu/127.9.123.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.125:36585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.123.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:08:16.556631 32213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:08:16.558028 32213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:08:16.570062 32219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:16.570094 32218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:16.570062 32221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:08:16.570952 32213 server_base.cc:1034] running on GCE node
I20260926 15:08:16.571758 32213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:08:16.573279 32213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:08:16.574535 32213 hybrid_clock.cc:648] HybridClock initialized: now 1790435296574463 us; error 43 us; skew 500 ppm
I20260926 15:08:16.574909 32213 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:08:16 dist-test-slave-7153 krb5kdc[32126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435296, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:08:16.578176 32213 init.cc:377] Logged in from keytab as kudu/127.9.123.125@KRBTEST.COM (short username kudu)
I20260926 15:08:16.579756 32213 webserver.cc:460] Webserver started at http://127.9.123.125:39367/ using document root <none> and password file <none>
I20260926 15:08:16.580454 32213 fs_manager.cc:359] Metadata directory not provided
I20260926 15:08:16.580559 32213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:08:16.580858 32213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:08:16.584024 32213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-1/data/instance:
uuid: "8284876e034e48faa44fb27a52031dbf"
format_stamp: "Formatted at 2026-09-26 15:08:16 on dist-test-slave-7153"
server_key: "1e28eb7b1f5a886aa7d58a65283559e6"
server_key_iv: "afda87ec0215ce50fe9c7ba5f7eff3d6"
server_key_version: "encryptionkey@0"
I20260926 15:08:16.584802 32213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-1/wal/instance:
uuid: "8284876e034e48faa44fb27a52031dbf"
format_stamp: "Formatted at 2026-09-26 15:08:16 on dist-test-slave-7153"
server_key: "1e28eb7b1f5a886aa7d58a65283559e6"
server_key_iv: "afda87ec0215ce50fe9c7ba5f7eff3d6"
server_key_version: "encryptionkey@0"
I20260926 15:08:16.590257 32213 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260926 15:08:16.593832 32228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:08:16.595361 32213 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:08:16.595551 32213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-1/wal
uuid: "8284876e034e48faa44fb27a52031dbf"
format_stamp: "Formatted at 2026-09-26 15:08:16 on dist-test-slave-7153"
server_key: "1e28eb7b1f5a886aa7d58a65283559e6"
server_key_iv: "afda87ec0215ce50fe9c7ba5f7eff3d6"
server_key_version: "encryptionkey@0"
I20260926 15:08:16.595762 32213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:08:16.609414 32213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:08:16.613483 32213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:08:16.613721 32213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:08:16.630963 32213 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.125:36585
I20260926 15:08:16.630995 32279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.125:36585 every 8 connection(s)
I20260926 15:08:16.632297 32213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-1/data/info.pb
I20260926 15:08:16.636287 32280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:08:16.639806 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 32213
I20260926 15:08:16.639992 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-1/wal/instance
I20260926 15:08:16.640506 9709 external_mini_cluster.cc:1428] Setting key 3402c1513570a2408dffa04f021f73cc
I20260926 15:08:16.644218 32280 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42713 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:08:16 dist-test-slave-7153 krb5kdc[32126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435296, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.125@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.123.124@KRBTEST.COM; defaulting to no policy
I20260926 15:08:16.665446 32210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.647436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40798 (local address 127.9.123.126:42713)
0926 15:08:16.647672 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:16.647679 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:16.648902 (+ 1223us) server_negotiation.cc:407] Connection header received
0926 15:08:16.650836 (+ 1934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:16.650843 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:16.650928 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:16.651052 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:16.652680 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:16.653405 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:16.654907 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:16.655312 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:16.655419 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:16.659102 (+ 3683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:16.659109 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:16.659114 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:16.659155 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:16.662095 (+ 2940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:16.663043 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:16.663047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:16.663049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:16.663168 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:16.663633 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:16.663637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:16.663639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:16.663920 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:16.664037 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:16.664971 (+ 934us) server_negotiation.cc:299] Negotiation successful
0926 15:08:16.665201 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 15:08:16.666440 32282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.647880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:42713 (local address 127.0.0.1:40798)
0926 15:08:16.648712 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.648765 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0926 15:08:16.650558 (+ 1793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:08:16.651235 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:08:16.651245 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:08:16.651792 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:08:16.652479 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:16.652498 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:16.653570 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:16.653575 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:16.654756 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:16.654768 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:16.655534 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:16.655542 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:16.655753 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:16.656423 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:08:16.656448 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:08:16.658830 (+ 2382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:08:16.662253 (+ 3423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:16.662264 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:16.662284 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:16.662936 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:16.663276 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:16.663281 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:16.663285 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:16.663525 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:16.664062 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:08:16.664072 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:08:16.664547 (+ 475us) client_negotiation.cc:769] Sending connection context
0926 15:08:16.664882 (+ 335us) client_negotiation.cc:240] Negotiation successful
0926 15:08:16.665351 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":368,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.124@KRBTEST.COM" created.
I20260926 15:08:16.668258 32280 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:08:16 dist-test-slave-7153 krb5kdc[32126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435296, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.125@KRBTEST.COM for kudu/127.9.123.125@KRBTEST.COM
I20260926 15:08:16.682480 32282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.668961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:36585 (local address 127.0.0.1:44934)
0926 15:08:16.669154 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.669169 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0926 15:08:16.669252 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:08:16.670392 (+ 1140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:08:16.670398 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:08:16.670800 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:08:16.670937 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:16.670945 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:16.672253 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:16.672258 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:16.673081 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:16.673091 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:16.673874 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:16.673880 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:16.674048 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:16.674568 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:08:16.674585 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:08:16.676687 (+ 2102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:08:16.679893 (+ 3206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:16.679897 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:16.679902 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:16.680354 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:16.680693 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:16.680697 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:16.680700 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:16.680869 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:16.681489 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:08:16.681493 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:08:16.681624 (+ 131us) client_negotiation.cc:769] Sending connection context
0926 15:08:16.681769 (+ 145us) client_negotiation.cc:240] Negotiation successful
0926 15:08:16.682060 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260926 15:08:16.682526 32284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.669258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44934 (local address 127.9.123.125:36585)
0926 15:08:16.669930 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:16.669936 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:16.669962 (+ 26us) server_negotiation.cc:407] Connection header received
0926 15:08:16.670006 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:16.670012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:16.670198 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:16.670347 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:16.671097 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:16.672124 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:16.673224 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:16.673740 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:16.673852 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:16.676933 (+ 3081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:16.676946 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:16.676960 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:16.676991 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:16.679748 (+ 2757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:16.680480 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:16.680487 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:16.680489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:16.680602 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:16.680970 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:16.680974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:16.680976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:16.681371 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:16.681523 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:16.681918 (+ 395us) server_negotiation.cc:299] Negotiation successful
0926 15:08:16.682168 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":508,"thread_start_us":370,"threads_started":1}
I20260926 15:08:16.685122 32280 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:08:16.686234 32282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.685787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:35867 (local address 127.0.0.1:36580)
0926 15:08:16.685964 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.686077 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260926 15:08:16.686559 32232 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
I20260926 15:08:16.688135 32282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.687801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:35867 (local address 127.0.0.1:36594)
0926 15:08:16.687952 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.688031 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 15:08:16.688342 32280 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:35867: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.124.keytab.
Entry for principal kudu/127.9.123.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.124.keytab.
I20260926 15:08:16.700251 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.124:35867
--webserver_interface=127.9.123.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.124.keytab
--principal=kudu/127.9.123.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:34837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:42713,127.9.123.125:36585,127.9.123.124:35867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:08:16.737665 32280 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } attempt: 1
I20260926 15:08:16.738853 32282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.738372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:35867 (local address 127.0.0.1:36608)
0926 15:08:16.738591 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.738701 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260926 15:08:16.739871 32282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.739481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:35867 (local address 127.0.0.1:36614)
0926 15:08:16.739660 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.739740 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260926 15:08:16.740067 32280 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:35867: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
I20260926 15:08:16.808331 32280 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } attempt: 2
I20260926 15:08:16.809551 32282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.809049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:35867 (local address 127.0.0.1:36628)
0926 15:08:16.809281 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.809378 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260926 15:08:16.810756 32282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.810352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:35867 (local address 127.0.0.1:36640)
0926 15:08:16.810538 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.810621 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260926 15:08:16.810966 32280 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:35867: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
I20260926 15:08:16.844023 32206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } attempt: 5
Sep 26 15:08:16 dist-test-slave-7153 krb5kdc[32126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435296, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for kudu/127.9.123.125@KRBTEST.COM
I20260926 15:08:16.858229 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.844957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:36585 (local address 127.0.0.1:44942)
0926 15:08:16.845180 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.845202 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 15:08:16.845336 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:08:16.845723 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:08:16.845728 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:08:16.846277 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:08:16.846477 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:16.846488 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:16.847519 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:16.847525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:16.848368 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:16.848380 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:16.849248 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:16.849255 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:16.849486 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:16.850092 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:08:16.850109 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:08:16.852563 (+ 2454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:08:16.855710 (+ 3147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:16.855716 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:16.855720 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:16.856110 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:16.856523 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:16.856529 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:16.856533 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:16.856675 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:16.857224 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:08:16.857230 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:08:16.857385 (+ 155us) client_negotiation.cc:769] Sending connection context
0926 15:08:16.857513 (+ 128us) client_negotiation.cc:240] Negotiation successful
0926 15:08:16.857875 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260926 15:08:16.858213 32284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.845203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44942 (local address 127.9.123.125:36585)
0926 15:08:16.845427 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:16.845433 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:16.845454 (+ 21us) server_negotiation.cc:407] Connection header received
0926 15:08:16.845518 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:16.845522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:16.845581 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:16.845667 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:16.846654 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:16.847379 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:16.848528 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:16.849078 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:16.849223 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:16.852816 (+ 3593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:16.852822 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:16.852826 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:16.852863 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:16.855535 (+ 2672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:16.856238 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:16.856243 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:16.856245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:16.856372 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:16.856780 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:16.856785 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:16.856787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:16.857080 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:16.857255 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:16.857643 (+ 388us) server_negotiation.cc:299] Negotiation successful
0926 15:08:16.857875 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 15:08:16.859469 32206 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:08:16.860682 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.860182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:35867 (local address 127.0.0.1:36650)
0926 15:08:16.860376 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.860479 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260926 15:08:16.861677 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.861296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:35867 (local address 127.0.0.1:36652)
0926 15:08:16.861433 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.861551 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260926 15:08:16.861886 32206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:35867: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
I20260926 15:08:16.902211 32280 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } attempt: 3
I20260926 15:08:16.903386 32282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.902880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:35867 (local address 127.0.0.1:36654)
0926 15:08:16.903108 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.903203 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260926 15:08:16.904626 32282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.904092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:35867 (local address 127.0.0.1:36662)
0926 15:08:16.904359 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.904489 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
W20260926 15:08:16.904834 32280 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:35867: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
I20260926 15:08:16.913166 32206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } attempt: 1
I20260926 15:08:16.914501 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.913952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:35867 (local address 127.0.0.1:36674)
0926 15:08:16.914182 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.914299 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260926 15:08:16.915522 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.915148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:35867 (local address 127.0.0.1:36676)
0926 15:08:16.915295 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.915383 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260926 15:08:16.915729 32206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:35867: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
I20260926 15:08:16.983960 32206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } attempt: 2
I20260926 15:08:16.985152 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.984559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:35867 (local address 127.0.0.1:36690)
0926 15:08:16.984802 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.984913 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260926 15:08:16.986287 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:16.985898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:35867 (local address 127.0.0.1:36700)
0926 15:08:16.986054 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:16.986158 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 15:08:16.986586 32206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:35867: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
W20260926 15:08:17.029608 32287 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 15:08:17.029973 32287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:08:17.030077 32287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:08:17.030193 32287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:08:17.037580 32287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:08:17.037696 32287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:08:17.037777 32287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:08:17.037842 32287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:08:17.037923 32287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:08:17.047920 32287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:34837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:42713,127.9.123.125:36585,127.9.123.124:35867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.124.keytab
--principal=kudu/127.9.123.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.124:35867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.123.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:08:17.048092 32280 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } attempt: 4
I20260926 15:08:17.049322 32282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:17.048765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:35867 (local address 127.0.0.1:36712)
0926 15:08:17.049002 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:17.049144 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260926 15:08:17.049525 32287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:08:17.050300 32282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:17.049955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:35867 (local address 127.0.0.1:36728)
0926 15:08:17.050111 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:17.050188 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260926 15:08:17.050516 32280 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:35867: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
I20260926 15:08:17.051088 32287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:08:17.062726 32295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:17.062862 32293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:17.063175 32292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:08:17.064177 32287 server_base.cc:1034] running on GCE node
I20260926 15:08:17.064985 32287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:08:17.066421 32287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:08:17.067668 32287 hybrid_clock.cc:648] HybridClock initialized: now 1790435297067595 us; error 47 us; skew 500 ppm
I20260926 15:08:17.068022 32287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:08:17 dist-test-slave-7153 krb5kdc[32126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435297, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:08:17.071285 32287 init.cc:377] Logged in from keytab as kudu/127.9.123.124@KRBTEST.COM (short username kudu)
I20260926 15:08:17.072837 32287 webserver.cc:460] Webserver started at http://127.9.123.124:38141/ using document root <none> and password file <none>
I20260926 15:08:17.073606 32287 fs_manager.cc:359] Metadata directory not provided
I20260926 15:08:17.073712 32287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:08:17.074023 32287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:08:17.077075 32287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-2/data/instance:
uuid: "29154f3056dc4a9eafcfbf3d5c563407"
format_stamp: "Formatted at 2026-09-26 15:08:17 on dist-test-slave-7153"
server_key: "d875a577c597769ff512699d5adb1985"
server_key_iv: "6effb229efbab824204c7c5955ff6271"
server_key_version: "encryptionkey@0"
I20260926 15:08:17.077926 32287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-2/wal/instance:
uuid: "29154f3056dc4a9eafcfbf3d5c563407"
format_stamp: "Formatted at 2026-09-26 15:08:17 on dist-test-slave-7153"
server_key: "d875a577c597769ff512699d5adb1985"
server_key_iv: "6effb229efbab824204c7c5955ff6271"
server_key_version: "encryptionkey@0"
I20260926 15:08:17.083581 32287 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260926 15:08:17.087566 32302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:08:17.089303 32287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260926 15:08:17.089507 32287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-2/wal
uuid: "29154f3056dc4a9eafcfbf3d5c563407"
format_stamp: "Formatted at 2026-09-26 15:08:17 on dist-test-slave-7153"
server_key: "d875a577c597769ff512699d5adb1985"
server_key_iv: "6effb229efbab824204c7c5955ff6271"
server_key_version: "encryptionkey@0"
I20260926 15:08:17.089736 32287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:08:17.092804 32206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } attempt: 3
I20260926 15:08:17.094005 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:17.093549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:35867 (local address 127.0.0.1:36740)
0926 15:08:17.093736 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:17.093823 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260926 15:08:17.095018 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:17.094596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:35867 (local address 127.0.0.1:36742)
0926 15:08:17.094788 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:17.094867 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260926 15:08:17.095261 32206 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:35867: Network error: Client connection negotiation failed: client connection to 127.9.123.124:35867: connect: Connection refused (error 111)
I20260926 15:08:17.118770 32287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:08:17.123205 32287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:08:17.123471 32287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:08:17.141542 32287 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.124:35867
I20260926 15:08:17.141594 32353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.124:35867 every 8 connection(s)
I20260926 15:08:17.142920 32287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-2/data/info.pb
I20260926 15:08:17.144627 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 32287
I20260926 15:08:17.144810 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790434939095847-9709-0/minicluster-data/master-2/wal/instance
I20260926 15:08:17.145360 9709 external_mini_cluster.cc:1428] Setting key f25f8f5defbd5cb5df3843b770f133af
I20260926 15:08:17.147710 32354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:08:17.157617 32354 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42713 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:08:17 dist-test-slave-7153 krb5kdc[32126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.124@KRBTEST.COM
Sep 26 15:08:17 dist-test-slave-7153 krb5kdc[32126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435297, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.124@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:08:17.171813 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:17.147219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36756 (local address 127.9.123.124:35867)
0926 15:08:17.148045 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:17.148067 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:17.148117 (+ 50us) server_negotiation.cc:407] Connection header received
0926 15:08:17.150645 (+ 2528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:17.150677 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:17.151294 (+ 617us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:17.151819 (+ 525us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:17.153515 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.155607 (+ 2092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.157038 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.158170 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.158394 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:17.163487 (+ 5093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:17.163517 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:17.163529 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:17.163574 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:17.167165 (+ 3591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:17.168096 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:17.168106 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:17.168116 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:17.168331 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:17.168723 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:17.168728 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:17.168732 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:17.169463 (+ 731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:17.169689 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:17.170264 (+ 575us) server_negotiation.cc:299] Negotiation successful
0926 15:08:17.170696 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":317,"threads_started":1}
I20260926 15:08:17.176663 32358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:17.160894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:42713 (local address 127.0.0.1:40808)
0926 15:08:17.161647 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:17.161687 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0926 15:08:17.161862 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:08:17.162278 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:08:17.162285 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:08:17.162989 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:08:17.163258 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:17.163268 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.164395 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.164401 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:17.166127 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:17.166152 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.167077 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.167084 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:17.167310 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:17.167922 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:08:17.167945 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:08:17.170464 (+ 2519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:08:17.173822 (+ 3358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:17.173831 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:17.173836 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:17.174338 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:17.174793 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:17.174798 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:17.174801 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:17.174963 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:17.175769 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:08:17.175775 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:08:17.175936 (+ 161us) client_negotiation.cc:769] Sending connection context
0926 15:08:17.176097 (+ 161us) client_negotiation.cc:240] Negotiation successful
0926 15:08:17.176356 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":361,"threads_started":1}
I20260926 15:08:17.176945 32210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:17.160878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40808 (local address 127.9.123.126:42713)
0926 15:08:17.161088 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:17.161118 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:17.161798 (+ 680us) server_negotiation.cc:407] Connection header received
0926 15:08:17.162011 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:17.162017 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:17.162093 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:17.162186 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:17.163420 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.164221 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.166321 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.166849 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.166964 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:17.170690 (+ 3726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:17.170696 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:17.170699 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:17.170736 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:17.173700 (+ 2964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:17.174496 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:17.174503 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:17.174506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:17.174685 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:17.175111 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:17.175117 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:17.175120 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:17.175603 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:17.175780 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:17.176256 (+ 476us) server_negotiation.cc:299] Negotiation successful
0926 15:08:17.176536 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260926 15:08:17.178323 32354 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:08:17 dist-test-slave-7153 krb5kdc[32126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
Sep 26 15:08:17 dist-test-slave-7153 krb5kdc[32126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435297, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.124@KRBTEST.COM for kudu/127.9.123.125@KRBTEST.COM
I20260926 15:08:17.192291 32358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:17.178924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:36585 (local address 127.0.0.1:44954)
0926 15:08:17.179059 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:17.179076 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0926 15:08:17.179162 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:08:17.179487 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:08:17.179491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:08:17.179827 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:08:17.179977 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:17.179984 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.180935 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.180940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:17.181847 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:17.181858 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.182747 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.182751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:17.182939 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:17.183429 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:08:17.183453 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:08:17.185743 (+ 2290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:08:17.189268 (+ 3525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:17.189273 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:17.189277 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:17.189845 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:17.190386 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:17.190391 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:17.190394 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:17.190623 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:17.191265 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:08:17.191270 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:08:17.191491 (+ 221us) client_negotiation.cc:769] Sending connection context
0926 15:08:17.191650 (+ 159us) client_negotiation.cc:240] Negotiation successful
0926 15:08:17.191952 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260926 15:08:17.192346 32210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:17.175138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40820 (local address 127.9.123.126:42713)
0926 15:08:17.177304 (+ 2166us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:17.177310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:17.177337 (+ 27us) server_negotiation.cc:407] Connection header received
0926 15:08:17.177392 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:17.177398 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:17.177474 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:17.177572 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:17.178919 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.179745 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.180936 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.181388 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.181497 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:17.185683 (+ 4186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:17.185689 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:17.185692 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:17.185725 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:17.189135 (+ 3410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:17.190005 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:17.190010 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:17.190014 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:17.190205 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:17.190739 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:17.190745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:17.190748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:17.191115 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:17.191370 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:17.191756 (+ 386us) server_negotiation.cc:299] Negotiation successful
0926 15:08:17.192011 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2050,"thread_start_us":2582,"threads_started":1}
I20260926 15:08:17.192360 32284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:17.179055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44954 (local address 127.9.123.125:36585)
0926 15:08:17.179236 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:17.179242 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:17.179258 (+ 16us) server_negotiation.cc:407] Connection header received
0926 15:08:17.179307 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:17.179310 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:17.179364 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:17.179450 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:17.180105 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.180805 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.181988 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.182587 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.182736 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:17.185939 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:17.185945 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:17.185950 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:17.185989 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:17.189057 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:17.190004 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:17.190010 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:17.190014 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:17.190205 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:17.190739 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:17.190745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:17.190749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:17.191120 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:17.191259 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:17.191765 (+ 506us) server_negotiation.cc:299] Negotiation successful
0926 15:08:17.191992 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 15:08:17.193329 32354 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:08:17 dist-test-slave-7153 krb5kdc[32126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435297, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.124@KRBTEST.COM for kudu/127.9.123.124@KRBTEST.COM
Sep 26 15:08:17 dist-test-slave-7153 krb5kdc[32126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.125@KRBTEST.COM
I20260926 15:08:17.208781 32358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:17.194283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:35867 (local address 127.0.0.1:36766)
0926 15:08:17.194541 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:17.194559 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:08:17.194703 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:08:17.196173 (+ 1470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:08:17.196176 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:08:17.196506 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:08:17.196641 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:17.196649 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.197712 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.197717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:17.198947 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:17.198960 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.199715 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.199720 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:17.199942 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:17.200512 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:08:17.200529 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:08:17.202719 (+ 2190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:08:17.206009 (+ 3290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:17.206015 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:17.206019 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:17.206533 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:17.206956 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:17.206959 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:17.206962 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:17.207094 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:17.207751 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:08:17.207756 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:08:17.207941 (+ 185us) client_negotiation.cc:769] Sending connection context
0926 15:08:17.208078 (+ 137us) client_negotiation.cc:240] Negotiation successful
0926 15:08:17.208377 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154}
I20260926 15:08:17.208856 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:17.194284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36766 (local address 127.9.123.124:35867)
0926 15:08:17.194543 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:17.194548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:17.194629 (+ 81us) server_negotiation.cc:407] Connection header received
0926 15:08:17.194805 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:17.194810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:17.194872 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:17.194977 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:17.196772 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.197579 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.199081 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.199564 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.199664 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:17.202915 (+ 3251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:17.202921 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:17.202926 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:17.202964 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:17.205848 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:17.206679 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:17.206683 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:17.206686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:17.206819 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:17.207197 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:17.207202 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:17.207205 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:17.207639 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:17.207751 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:17.208199 (+ 448us) server_negotiation.cc:299] Negotiation successful
0926 15:08:17.208560 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":125,"server-negotiator.queue_time_us":156}
I20260926 15:08:17.209713 32284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:17.194568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44970 (local address 127.9.123.125:36585)
0926 15:08:17.194728 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:17.194731 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:17.194746 (+ 15us) server_negotiation.cc:407] Connection header received
0926 15:08:17.194793 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:17.194798 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:17.194864 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:17.194963 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:17.196140 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.196939 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.198570 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.199023 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.199144 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:17.203879 (+ 4735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:17.203887 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:17.203892 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:17.203938 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:17.206776 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:17.207653 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:17.207658 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:17.207660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:17.207834 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:17.208270 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:17.208276 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:17.208280 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:17.208635 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:17.208920 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:17.209224 (+ 304us) server_negotiation.cc:299] Negotiation successful
0926 15:08:17.209415 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 15:08:17.214550 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:08:17.214804 32354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29154f3056dc4a9eafcfbf3d5c563407: Bootstrap starting.
I20260926 15:08:17.218268 32354 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29154f3056dc4a9eafcfbf3d5c563407: Neither blocks nor log segments found. Creating new log.
I20260926 15:08:17.219594 32354 log.cc:824] T 00000000000000000000000000000000 P 29154f3056dc4a9eafcfbf3d5c563407: Log is configured to *not* fsync() on all Append() calls
I20260926 15:08:17.217200 32181 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40820
I20260926 15:08:17.223278 32354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29154f3056dc4a9eafcfbf3d5c563407: No bootstrap required, opened a new log
I20260926 15:08:17.231562 32354 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29154f3056dc4a9eafcfbf3d5c563407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42713 } } peers { permanent_uuid: "8284876e034e48faa44fb27a52031dbf" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } } peers { permanent_uuid: "29154f3056dc4a9eafcfbf3d5c563407" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } }
I20260926 15:08:17.231950 32354 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29154f3056dc4a9eafcfbf3d5c563407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29154f3056dc4a9eafcfbf3d5c563407, State: Initialized, Role: FOLLOWER
I20260926 15:08:17.232456 32354 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29154f3056dc4a9eafcfbf3d5c563407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42713 } } peers { permanent_uuid: "8284876e034e48faa44fb27a52031dbf" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } } peers { permanent_uuid: "29154f3056dc4a9eafcfbf3d5c563407" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } }
I20260926 15:08:17.233600 32361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29154f3056dc4a9eafcfbf3d5c563407 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42713 } } peers { permanent_uuid: "8284876e034e48faa44fb27a52031dbf" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } } peers { permanent_uuid: "29154f3056dc4a9eafcfbf3d5c563407" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } } }
I20260926 15:08:17.234165 32361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29154f3056dc4a9eafcfbf3d5c563407 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:08:17.234638 32354 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29154f3056dc4a9eafcfbf3d5c563407 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 15:08:17.245996 32372 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 29154f3056dc4a9eafcfbf3d5c563407: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 15:08:17.246281 32372 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 15:08:17.243043 32287 master_runner.cc:186] Error getting master registration for 127.9.123.126:42713: OK, instance_id { permanent_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" instance_seqno: 1790435296129881 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 15:08:17.252287 32287 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:08:17.272531 32206 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } attempt: 4
Sep 26 15:08:17 dist-test-slave-7153 krb5kdc[32126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435296, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for kudu/127.9.123.124@KRBTEST.COM
I20260926 15:08:17.287127 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:17.273413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36782 (local address 127.9.123.124:35867)
0926 15:08:17.273655 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:17.273661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:17.273685 (+ 24us) server_negotiation.cc:407] Connection header received
0926 15:08:17.273843 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:17.273848 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:17.273913 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:17.274009 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:17.274831 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.275659 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.276901 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.277415 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.277529 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:17.280992 (+ 3463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:17.280998 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:17.281001 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:17.281036 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:17.284527 (+ 3491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:17.285190 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:17.285196 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:17.285200 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:17.285324 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:17.285698 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:17.285708 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:17.285711 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:17.286036 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:17.286188 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:17.286562 (+ 374us) server_negotiation.cc:299] Negotiation successful
0926 15:08:17.286765 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260926 15:08:17.287117 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:17.273343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:35867 (local address 127.0.0.1:36782)
0926 15:08:17.273537 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:17.273558 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 15:08:17.273675 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:08:17.274057 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:08:17.274062 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:08:17.274493 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:08:17.274668 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:17.274678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.275805 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.275811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:17.276753 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:17.276765 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.277563 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.277569 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:17.277769 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:17.278349 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:08:17.278368 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:08:17.280778 (+ 2410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:08:17.284659 (+ 3881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:17.284665 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:17.284670 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:17.285064 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:17.285433 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:17.285438 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:17.285442 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:17.285596 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:17.286166 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:08:17.286171 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:08:17.286323 (+ 152us) client_negotiation.cc:769] Sending connection context
0926 15:08:17.286445 (+ 122us) client_negotiation.cc:240] Negotiation successful
0926 15:08:17.286744 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260926 15:08:17.291525 32206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4: Bootstrap starting.
I20260926 15:08:17.295331 32206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4: Neither blocks nor log segments found. Creating new log.
I20260926 15:08:17.296429 32206 log.cc:824] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4: Log is configured to *not* fsync() on all Append() calls
I20260926 15:08:17.299356 32206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4: No bootstrap required, opened a new log
I20260926 15:08:17.304539 32206 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42713 } } peers { permanent_uuid: "8284876e034e48faa44fb27a52031dbf" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } } peers { permanent_uuid: "29154f3056dc4a9eafcfbf3d5c563407" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } }
I20260926 15:08:17.304874 32206 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccfbcab13eff4bf69d235f6c99bfe8c4, State: Initialized, Role: FOLLOWER
I20260926 15:08:17.305356 32206 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42713 } } peers { permanent_uuid: "8284876e034e48faa44fb27a52031dbf" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } } peers { permanent_uuid: "29154f3056dc4a9eafcfbf3d5c563407" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } }
I20260926 15:08:17.306247 32375 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42713 } } peers { permanent_uuid: "8284876e034e48faa44fb27a52031dbf" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } } peers { permanent_uuid: "29154f3056dc4a9eafcfbf3d5c563407" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } } }
I20260926 15:08:17.306634 32375 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:08:17.307416 32206 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:08:17.307574 32181 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 15:08:17.307775 32181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:08:17.311226 32181 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42713 } } peers { permanent_uuid: "8284876e034e48faa44fb27a52031dbf" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } } peers { permanent_uuid: "29154f3056dc4a9eafcfbf3d5c563407" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } }
I20260926 15:08:17.313865 32181 leader_election.cc:288] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4 [CANDIDATE]: Term 1 election: Requested vote from peers 8284876e034e48faa44fb27a52031dbf (127.9.123.125:36585), 29154f3056dc4a9eafcfbf3d5c563407 (127.9.123.124:35867)
I20260926 15:08:17.313958 32329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "29154f3056dc4a9eafcfbf3d5c563407"
I20260926 15:08:17.314467 32329 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29154f3056dc4a9eafcfbf3d5c563407 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:08:17.317967 32329 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 29154f3056dc4a9eafcfbf3d5c563407 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfbcab13eff4bf69d235f6c99bfe8c4 in term 1.
I20260926 15:08:17.318683 32158 leader_election.cc:302] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29154f3056dc4a9eafcfbf3d5c563407, ccfbcab13eff4bf69d235f6c99bfe8c4; no voters:
I20260926 15:08:17.319170 32375 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:08:17.330884 32375 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4 [term 1 LEADER]: Becoming Leader. State: Replica: ccfbcab13eff4bf69d235f6c99bfe8c4, State: Running, Role: LEADER
I20260926 15:08:17.331635 32375 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42713 } } peers { permanent_uuid: "8284876e034e48faa44fb27a52031dbf" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } } peers { permanent_uuid: "29154f3056dc4a9eafcfbf3d5c563407" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } }
I20260926 15:08:17.346436 32384 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccfbcab13eff4bf69d235f6c99bfe8c4. Latest consensus state: current_term: 1 leader_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42713 } } peers { permanent_uuid: "8284876e034e48faa44fb27a52031dbf" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } } peers { permanent_uuid: "29154f3056dc4a9eafcfbf3d5c563407" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } } }
I20260926 15:08:17.346971 32384 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4 [sys.catalog]: This master's current role is: LEADER
I20260926 15:08:17.349690 32280 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } attempt: 5
I20260926 15:08:17.357730 32389 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 26 15:08:17 dist-test-slave-7153 krb5kdc[32126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435296, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.125@KRBTEST.COM for kudu/127.9.123.124@KRBTEST.COM
I20260926 15:08:17.368778 32389 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:08:17.369966 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:17.351256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36790 (local address 127.9.123.124:35867)
0926 15:08:17.351452 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:17.351458 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:17.351479 (+ 21us) server_negotiation.cc:407] Connection header received
0926 15:08:17.351547 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:17.351553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:17.351631 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:17.351732 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:17.353041 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.354294 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.356288 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.357024 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.357167 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:17.361865 (+ 4698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:17.361873 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:17.361878 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:17.361927 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:17.365868 (+ 3941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:17.366877 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:17.366884 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:17.366889 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:17.367078 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:17.367569 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:17.367575 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:17.367578 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:17.368513 (+ 935us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:17.368666 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:17.369287 (+ 621us) server_negotiation.cc:299] Negotiation successful
0926 15:08:17.369576 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260926 15:08:17.369983 32160 tablet.cc:2369] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:08:17.371028 32282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:17.350385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:35867 (local address 127.0.0.1:36790)
0926 15:08:17.350822 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:17.350844 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0926 15:08:17.351010 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:08:17.351991 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:08:17.351998 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:08:17.352549 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:08:17.352778 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:17.352790 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.354526 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.354533 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:17.356052 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:17.356070 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.357464 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.357471 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:17.357716 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:17.358416 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:08:17.358440 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:08:17.361600 (+ 3160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:08:17.366108 (+ 4508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:17.366114 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:17.366119 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:17.366716 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:17.367215 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:17.367221 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:17.367225 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:17.367434 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:17.368756 (+ 1322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:08:17.368762 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:08:17.368934 (+ 172us) client_negotiation.cc:769] Sending connection context
0926 15:08:17.370203 (+ 1269us) client_negotiation.cc:240] Negotiation successful
0926 15:08:17.370608 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295}
W20260926 15:08:17.375656 32308 tablet.cc:2369] T 00000000000000000000000000000000 P 29154f3056dc4a9eafcfbf3d5c563407: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:08:17.377233 32280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8284876e034e48faa44fb27a52031dbf: Bootstrap starting.
I20260926 15:08:17.382853 32280 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8284876e034e48faa44fb27a52031dbf: Neither blocks nor log segments found. Creating new log.
I20260926 15:08:17.384519 32280 log.cc:824] T 00000000000000000000000000000000 P 8284876e034e48faa44fb27a52031dbf: Log is configured to *not* fsync() on all Append() calls
I20260926 15:08:17.409932 32329 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 29154f3056dc4a9eafcfbf3d5c563407 [term 1 FOLLOWER]: Refusing update from remote peer ccfbcab13eff4bf69d235f6c99bfe8c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:08:17.411279 32375 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29154f3056dc4a9eafcfbf3d5c563407" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:08:17.431507 32280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8284876e034e48faa44fb27a52031dbf: No bootstrap required, opened a new log
I20260926 15:08:17.439723 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:17.379918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36792 (local address 127.9.123.124:35867)
0926 15:08:17.380116 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:17.380121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:17.397939 (+ 17818us) server_negotiation.cc:407] Connection header received
0926 15:08:17.399966 (+ 2027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:17.399973 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:17.400079 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:17.400156 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:17.402597 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.403656 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.417386 (+ 13730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.418052 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.418158 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:17.426019 (+ 7861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:17.426027 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:17.426031 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:17.426078 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:17.433298 (+ 7220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:17.434905 (+ 1607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:17.434912 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:17.434916 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:17.435327 (+ 411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:17.436142 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:17.436147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:17.436151 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:17.436946 (+ 795us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:17.437416 (+ 470us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:17.438445 (+ 1029us) server_negotiation.cc:299] Negotiation successful
0926 15:08:17.438955 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 15:08:17.435328 32284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:17.398556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44982 (local address 127.9.123.125:36585)
0926 15:08:17.398744 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:17.398750 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:17.399549 (+ 799us) server_negotiation.cc:407] Connection header received
0926 15:08:17.403932 (+ 4383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:17.403938 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:17.404034 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:17.404107 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:17.413244 (+ 9137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.414328 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.419233 (+ 4905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.419889 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.419993 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:17.427575 (+ 7582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:17.427583 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:17.427588 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:17.427630 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:17.431404 (+ 3774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:17.432286 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:17.432293 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:17.432297 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:17.432486 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:17.432981 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:17.432987 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:17.432990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:17.433422 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:17.433666 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:17.434531 (+ 865us) server_negotiation.cc:299] Negotiation successful
0926 15:08:17.434775 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260926 15:08:17.453223 32400 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 15:08:17.453465 32361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29154f3056dc4a9eafcfbf3d5c563407 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccfbcab13eff4bf69d235f6c99bfe8c4. Latest consensus state: current_term: 1 leader_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42713 } } peers { permanent_uuid: "8284876e034e48faa44fb27a52031dbf" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } } peers { permanent_uuid: "29154f3056dc4a9eafcfbf3d5c563407" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } } }
I20260926 15:08:17.453799 32361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29154f3056dc4a9eafcfbf3d5c563407 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:08:17.455646 32393 mvcc.cc:205] Tried to move back new op lower bound from 7333622978117652480 to 7333622977877450752. Current Snapshot: MvccSnapshot[applied={T|T < 7333622978117652480}]
I20260926 15:08:17.457026 32280 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8284876e034e48faa44fb27a52031dbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42713 } } peers { permanent_uuid: "8284876e034e48faa44fb27a52031dbf" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } } peers { permanent_uuid: "29154f3056dc4a9eafcfbf3d5c563407" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } }
I20260926 15:08:17.457589 32280 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8284876e034e48faa44fb27a52031dbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8284876e034e48faa44fb27a52031dbf, State: Initialized, Role: FOLLOWER
I20260926 15:08:17.458170 32280 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8284876e034e48faa44fb27a52031dbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42713 } } peers { permanent_uuid: "8284876e034e48faa44fb27a52031dbf" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } } peers { permanent_uuid: "29154f3056dc4a9eafcfbf3d5c563407" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } }
I20260926 15:08:17.459125 32399 mvcc.cc:205] Tried to move back new op lower bound from 7333622978117652480 to 7333622977877450752. Current Snapshot: MvccSnapshot[applied={T|T < 7333622978117652480}]
I20260926 15:08:17.461638 32375 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42713 } } peers { permanent_uuid: "8284876e034e48faa44fb27a52031dbf" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } } peers { permanent_uuid: "29154f3056dc4a9eafcfbf3d5c563407" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } } }
I20260926 15:08:17.461962 32375 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4 [sys.catalog]: This master's current role is: LEADER
I20260926 15:08:17.465821 32361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29154f3056dc4a9eafcfbf3d5c563407 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42713 } } peers { permanent_uuid: "8284876e034e48faa44fb27a52031dbf" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } } peers { permanent_uuid: "29154f3056dc4a9eafcfbf3d5c563407" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } } }
I20260926 15:08:17.466137 32361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29154f3056dc4a9eafcfbf3d5c563407 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:08:17.466775 32389 catalog_manager.cc:1347] Generated new cluster ID: 1f206f52d9a04bbbaae46845de57ca69
I20260926 15:08:17.466892 32389 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:08:17.472098 32210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:17.379570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40834 (local address 127.9.123.126:42713)
0926 15:08:17.379773 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:17.379780 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:17.396657 (+ 16877us) server_negotiation.cc:407] Connection header received
0926 15:08:17.396912 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:17.396919 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:17.397036 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:17.397179 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:17.402276 (+ 5097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.403536 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.424978 (+ 21442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:17.425741 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:17.425858 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:17.446300 (+ 20442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:17.446307 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:17.446312 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:17.446358 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:17.450236 (+ 3878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:17.451764 (+ 1528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:17.451769 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:17.451772 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:17.451946 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:17.466683 (+ 14737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:17.466691 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:17.466695 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:17.469640 (+ 2945us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:17.469988 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:17.471196 (+ 1208us) server_negotiation.cc:299] Negotiation successful
0926 15:08:17.471534 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 15:08:17.482687 32403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8284876e034e48faa44fb27a52031dbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42713 } } peers { permanent_uuid: "8284876e034e48faa44fb27a52031dbf" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } } peers { permanent_uuid: "29154f3056dc4a9eafcfbf3d5c563407" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } } }
I20260926 15:08:17.483263 32403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8284876e034e48faa44fb27a52031dbf [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:08:17.484133 32280 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8284876e034e48faa44fb27a52031dbf [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:08:17.484567 32254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8284876e034e48faa44fb27a52031dbf [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:08:17.488257 32254 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8284876e034e48faa44fb27a52031dbf [term 1 FOLLOWER]: Refusing update from remote peer ccfbcab13eff4bf69d235f6c99bfe8c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:08:17.488996 32255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8284876e034e48faa44fb27a52031dbf"
I20260926 15:08:17.489789 32384 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8284876e034e48faa44fb27a52031dbf" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:08:17.492475 32255 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8284876e034e48faa44fb27a52031dbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccfbcab13eff4bf69d235f6c99bfe8c4 in term 1.
I20260926 15:08:17.464495 32139 master_runner.cc:186] Error getting master registration for 127.9.123.125:36585: OK, instance_id { permanent_uuid: "8284876e034e48faa44fb27a52031dbf" instance_seqno: 1790435296626073 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 15:08:17.499944 32139 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:08:17.521832 32389 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:08:17.535975 32389 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:08:17.558182 32403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8284876e034e48faa44fb27a52031dbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccfbcab13eff4bf69d235f6c99bfe8c4. Latest consensus state: current_term: 1 leader_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42713 } } peers { permanent_uuid: "8284876e034e48faa44fb27a52031dbf" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } } peers { permanent_uuid: "29154f3056dc4a9eafcfbf3d5c563407" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } } }
I20260926 15:08:17.558547 32403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8284876e034e48faa44fb27a52031dbf [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:08:17.558948 32410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8284876e034e48faa44fb27a52031dbf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42713 } } peers { permanent_uuid: "8284876e034e48faa44fb27a52031dbf" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } } peers { permanent_uuid: "29154f3056dc4a9eafcfbf3d5c563407" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } } }
I20260926 15:08:17.559293 32410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8284876e034e48faa44fb27a52031dbf [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:08:17.563010 32375 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccfbcab13eff4bf69d235f6c99bfe8c4" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 42713 } } peers { permanent_uuid: "8284876e034e48faa44fb27a52031dbf" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 36585 } } peers { permanent_uuid: "29154f3056dc4a9eafcfbf3d5c563407" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 35867 } } }
I20260926 15:08:17.563302 32375 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4 [sys.catalog]: This master's current role is: LEADER
I20260926 15:08:17.581485 32389 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccfbcab13eff4bf69d235f6c99bfe8c4: Generated new TSK 0
I20260926 15:08:17.582679 32389 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:08:17.593837 32420 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260926 15:08:17.676918 32210 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:17.668153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40848 (local address 127.9.123.126:42713)
0926 15:08:17.668356 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:17.668362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:17.674695 (+ 6333us) server_negotiation.cc:407] Connection header received
0926 15:08:17.675214 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:17.675221 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:17.675329 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:17.675413 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:17.676642 (+ 1229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40848: BlockingRecv error: recv got EOF from 127.0.0.1:40848 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260926 15:08:17.679208 32357 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:17.667498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36800 (local address 127.9.123.124:35867)
0926 15:08:17.667898 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:17.667904 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:17.677606 (+ 9702us) server_negotiation.cc:407] Connection header received
0926 15:08:17.677751 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:17.677756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:17.677858 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:17.677978 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:17.678961 (+ 983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36800: BlockingRecv error: recv got EOF from 127.0.0.1:36800 (error 108)
Metrics: {"server-negotiator.queue_time_us":239}
W20260926 15:08:17.681378 32284 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:17.678787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44992 (local address 127.9.123.125:36585)
0926 15:08:17.678968 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:17.678974 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:17.679569 (+ 595us) server_negotiation.cc:407] Connection header received
0926 15:08:17.679846 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:17.679852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:17.679930 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:17.679995 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:17.681156 (+ 1161us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44992: BlockingRecv error: recv got EOF from 127.0.0.1:44992 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260926 15:08:17.688534 32284 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:17.686182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45002 (local address 127.9.123.125:36585)
0926 15:08:17.686375 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:17.686381 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:17.686910 (+ 529us) server_negotiation.cc:407] Connection header received
0926 15:08:17.687155 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:17.687162 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:17.687309 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:17.687427 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:17.688258 (+ 831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45002: BlockingRecv error: recv got EOF from 127.0.0.1:45002 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260926 15:08:17.695631 32284 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:17.693364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45010 (local address 127.9.123.125:36585)
0926 15:08:17.693562 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:17.693568 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:17.694130 (+ 562us) server_negotiation.cc:407] Connection header received
0926 15:08:17.694340 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:17.694346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:17.694441 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:17.694544 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:17.695421 (+ 877us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45010: BlockingRecv error: recv got EOF from 127.0.0.1:45010 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260926 15:08:17.703368 32284 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:17.701067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45026 (local address 127.9.123.125:36585)
0926 15:08:17.701269 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:17.701275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:17.701795 (+ 520us) server_negotiation.cc:407] Connection header received
0926 15:08:17.702018 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:17.702024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:17.702139 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:17.702260 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:17.703112 (+ 852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45026: BlockingRecv error: recv got EOF from 127.0.0.1:45026 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260926 15:08:17.721666 32284 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:17.719327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45030 (local address 127.9.123.125:36585)
0926 15:08:17.719568 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:17.719574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:17.720113 (+ 539us) server_negotiation.cc:407] Connection header received
0926 15:08:17.720321 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:17.720327 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:17.720434 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:17.720561 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:17.721415 (+ 854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45030: BlockingRecv error: recv got EOF from 127.0.0.1:45030 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
W20260926 15:08:17.736568 32284 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:17.734232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45036 (local address 127.9.123.125:36585)
0926 15:08:17.734426 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:17.734433 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:17.735026 (+ 593us) server_negotiation.cc:407] Connection header received
0926 15:08:17.735218 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:17.735224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:17.735325 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:17.735432 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:17.736316 (+ 884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45036: BlockingRecv error: recv got EOF from 127.0.0.1:45036 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260926 15:08:17.758996 32284 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:17.756565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45048 (local address 127.9.123.125:36585)
0926 15:08:17.756759 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:17.756766 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:17.757393 (+ 627us) server_negotiation.cc:407] Connection header received
0926 15:08:17.757612 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:17.757619 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:17.757720 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:17.757855 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:17.758735 (+ 880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45048: BlockingRecv error: recv got EOF from 127.0.0.1:45048 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260926 15:08:17.797530 32284 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:17.795047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45056 (local address 127.9.123.125:36585)
0926 15:08:17.795236 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:17.795243 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:17.795888 (+ 645us) server_negotiation.cc:407] Connection header received
0926 15:08:17.796092 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:17.796099 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:17.796208 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:17.796343 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:17.797254 (+ 911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45056: BlockingRecv error: recv got EOF from 127.0.0.1:45056 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260926 15:08:17.867353 32284 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:17.865255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45058 (local address 127.9.123.125:36585)
0926 15:08:17.865511 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:17.865517 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:17.865924 (+ 407us) server_negotiation.cc:407] Connection header received
0926 15:08:17.866126 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:17.866131 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:17.866202 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:17.866289 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:17.867123 (+ 834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45058: BlockingRecv error: recv got EOF from 127.0.0.1:45058 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260926 15:08:18.000823 32284 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:17.999137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45062 (local address 127.9.123.125:36585)
0926 15:08:17.999352 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:17.999359 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:17.999552 (+ 193us) server_negotiation.cc:407] Connection header received
0926 15:08:17.999721 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:17.999725 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:17.999798 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:17.999885 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:18.000586 (+ 701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45062: BlockingRecv error: recv got EOF from 127.0.0.1:45062 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260926 15:08:18.248373 32372 catalog_manager.cc:1259] Loaded cluster ID: 1f206f52d9a04bbbaae46845de57ca69
I20260926 15:08:18.248486 32372 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 29154f3056dc4a9eafcfbf3d5c563407: loading cluster ID for follower catalog manager: success
I20260926 15:08:18.251286 32372 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 29154f3056dc4a9eafcfbf3d5c563407: acquiring CA information for follower catalog manager: success
I20260926 15:08:18.253402 32372 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 29154f3056dc4a9eafcfbf3d5c563407: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260926 15:08:18.263176 32284 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:18.261400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45070 (local address 127.9.123.125:36585)
0926 15:08:18.261621 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:18.261627 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:18.261784 (+ 157us) server_negotiation.cc:407] Connection header received
0926 15:08:18.261990 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:18.261995 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:18.262069 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:18.262157 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:18.262966 (+ 809us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45070: BlockingRecv error: recv got EOF from 127.0.0.1:45070 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260926 15:08:18.598345 32420 catalog_manager.cc:1259] Loaded cluster ID: 1f206f52d9a04bbbaae46845de57ca69
I20260926 15:08:18.598533 32420 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8284876e034e48faa44fb27a52031dbf: loading cluster ID for follower catalog manager: success
I20260926 15:08:18.601404 32420 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8284876e034e48faa44fb27a52031dbf: acquiring CA information for follower catalog manager: success
I20260926 15:08:18.603231 32420 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8284876e034e48faa44fb27a52031dbf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 15:08:18.792030 32485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:18.784784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45082 (local address 127.9.123.125:36585)
0926 15:08:18.785613 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:18.785619 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:18.785641 (+ 22us) server_negotiation.cc:407] Connection header received
0926 15:08:18.785722 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:18.785728 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:18.785931 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:18.786079 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:08:18.786557 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:18.787849 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:18.789215 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:18.789913 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:18.790050 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:18.790611 (+ 561us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:08:18.791352 (+ 741us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:08:18.791516 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:18.791566 (+ 50us) server_negotiation.cc:299] Negotiation successful
0926 15:08:18.791657 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":695,"thread_start_us":529,"threads_started":1}
I20260926 15:08:18.805179 32491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:18.798506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36802 (local address 127.9.123.124:35867)
0926 15:08:18.799444 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:18.799449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:18.799463 (+ 14us) server_negotiation.cc:407] Connection header received
0926 15:08:18.799550 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:18.799554 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:18.799714 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:18.799853 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:08:18.800286 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:18.800992 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:18.802883 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:18.803288 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:18.803399 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:18.803983 (+ 584us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:08:18.804539 (+ 556us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:08:18.804654 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:18.804778 (+ 124us) server_negotiation.cc:299] Negotiation successful
0926 15:08:18.804854 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":843,"thread_start_us":756,"threads_started":1}
I20260926 15:08:18.836126 32485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:18.824793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45090 (local address 127.9.123.125:36585)
0926 15:08:18.824999 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:18.825005 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:18.828992 (+ 3987us) server_negotiation.cc:407] Connection header received
0926 15:08:18.829294 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:18.829299 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:18.829386 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:18.829462 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:08:18.830302 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:18.831367 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:18.833812 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:18.834456 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:18.834548 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:18.835268 (+ 720us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:08:18.835409 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:08:18.835489 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:18.835703 (+ 214us) server_negotiation.cc:299] Negotiation successful
0926 15:08:18.835790 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260926 15:08:18.836256 32491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:18.812989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36808 (local address 127.9.123.124:35867)
0926 15:08:18.813204 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:18.813212 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:18.828545 (+ 15333us) server_negotiation.cc:407] Connection header received
0926 15:08:18.828953 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:18.828959 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:18.829053 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:18.829144 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:08:18.830175 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:18.831244 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:18.833832 (+ 2588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:18.834498 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:18.834624 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:18.835386 (+ 762us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:08:18.835513 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:08:18.835573 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:18.835884 (+ 311us) server_negotiation.cc:299] Negotiation successful
0926 15:08:18.835970 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260926 15:08:18.836621 32497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:18.812771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40860 (local address 127.9.123.126:42713)
0926 15:08:18.828595 (+ 15824us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:18.828601 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:18.828618 (+ 17us) server_negotiation.cc:407] Connection header received
0926 15:08:18.828750 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:18.828756 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:18.829013 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:18.829954 (+ 941us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:08:18.830129 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:18.831220 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:18.833817 (+ 2597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:18.834442 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:18.834565 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:18.835306 (+ 741us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:08:18.835749 (+ 443us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:08:18.835828 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:18.836264 (+ 436us) server_negotiation.cc:299] Negotiation successful
0926 15:08:18.836348 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15696,"thread_start_us":15940,"threads_started":1}
I20260926 15:08:18.841943 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 32139
I20260926 15:08:18.843364 32201 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:08:18.969679 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 32139
I20260926 15:08:18.986886 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 32213
I20260926 15:08:18.988003 32275 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:08:19.097232 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 32213
I20260926 15:08:19.113715 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 32287
I20260926 15:08:19.114806 32349 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:08:19.221091 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 32287
2026-09-26T15:08:19Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4821 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 26 15:08:19 dist-test-slave-7153 krb5kdc[32505](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 26 15:08:19 dist-test-slave-7153 krb5kdc[32505](info): set up 2 sockets
Sep 26 15:08:19 dist-test-slave-7153 krb5kdc[32505](info): commencing operation
krb5kdc: starting...
W20260926 15:08:20.273576 9709 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.009s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/krb5kdc/test-user.keytab.
Sep 26 15:08:20 dist-test-slave-7153 krb5kdc[32505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-26T15:08:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-26T15:08:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.123.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
Entry for principal kudu/127.9.123.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab.
I20260926 15:08:20.532032 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:39155
--webserver_interface=127.9.123.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:43093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:39155,127.9.123.125:42053,127.9.123.124:36169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260926 15:08:20.857764 32518 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 15:08:20.858134 32518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:08:20.858235 32518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:08:20.858341 32518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:08:20.865917 32518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:08:20.866015 32518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:08:20.866075 32518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:08:20.866155 32518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:08:20.866235 32518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:08:20.876220 32518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:43093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:39155,127.9.123.125:42053,127.9.123.124:36169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.126.keytab
--principal=kudu/127.9.123.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.126:39155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.123.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:08:20.877681 32518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:08:20.879092 32518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:08:20.890121 32523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:20.890725 32526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:20.891062 32524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:08:20.891986 32518 server_base.cc:1034] running on GCE node
I20260926 15:08:20.892716 32518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:08:20.894130 32518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:08:20.895354 32518 hybrid_clock.cc:648] HybridClock initialized: now 1790435300895295 us; error 31 us; skew 500 ppm
I20260926 15:08:20.895670 32518 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:08:20 dist-test-slave-7153 krb5kdc[32505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435300, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:08:20.898692 32518 init.cc:377] Logged in from keytab as kudu/127.9.123.126@KRBTEST.COM (short username kudu)
I20260926 15:08:20.900101 32518 webserver.cc:460] Webserver started at http://127.9.123.126:37275/ using document root <none> and password file <none>
I20260926 15:08:20.900801 32518 fs_manager.cc:359] Metadata directory not provided
I20260926 15:08:20.900898 32518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:08:20.901242 32518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:08:20.904125 32518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-0/data/instance:
uuid: "14e19504a2ad4639a3c9f3ce785e8176"
format_stamp: "Formatted at 2026-09-26 15:08:20 on dist-test-slave-7153"
server_key: "d243b8d7abdc52c4b2401095932ccbb5"
server_key_iv: "5a7cc2ec6c5ebad4deee861f2bc16046"
server_key_version: "encryptionkey@0"
I20260926 15:08:20.904896 32518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-0/wal/instance:
uuid: "14e19504a2ad4639a3c9f3ce785e8176"
format_stamp: "Formatted at 2026-09-26 15:08:20 on dist-test-slave-7153"
server_key: "d243b8d7abdc52c4b2401095932ccbb5"
server_key_iv: "5a7cc2ec6c5ebad4deee861f2bc16046"
server_key_version: "encryptionkey@0"
I20260926 15:08:20.910229 32518 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260926 15:08:20.913812 32533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:08:20.915308 32518 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260926 15:08:20.915481 32518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-0/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-0/wal
uuid: "14e19504a2ad4639a3c9f3ce785e8176"
format_stamp: "Formatted at 2026-09-26 15:08:20 on dist-test-slave-7153"
server_key: "d243b8d7abdc52c4b2401095932ccbb5"
server_key_iv: "5a7cc2ec6c5ebad4deee861f2bc16046"
server_key_version: "encryptionkey@0"
I20260926 15:08:20.915699 32518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:08:20.935402 32518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:08:20.939471 32518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:08:20.939734 32518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:08:20.956959 32518 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.126:39155
I20260926 15:08:20.956966 32584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.126:39155 every 8 connection(s)
I20260926 15:08:20.958432 32518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-0/data/info.pb
I20260926 15:08:20.962080 32585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:08:20.966174 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 32518
I20260926 15:08:20.966346 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-0/wal/instance
I20260926 15:08:20.966853 9709 external_mini_cluster.cc:1428] Setting key f86992fd81f678ee986a3abfb906e19f
I20260926 15:08:20.969879 32585 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39155 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:08:20 dist-test-slave-7153 krb5kdc[32505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435300, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.123.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.125@KRBTEST.COM" created.
I20260926 15:08:20.993134 32588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:20.973563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39808 (local address 127.9.123.126:39155)
0926 15:08:20.975376 (+ 1813us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:20.975383 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:20.975433 (+ 50us) server_negotiation.cc:407] Connection header received
0926 15:08:20.977295 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:20.977301 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:20.977556 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:20.977732 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:20.979028 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:20.980226 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:20.981687 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:20.982259 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:20.982449 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:20.986049 (+ 3600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:20.986059 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:20.986073 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:20.986105 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:20.988832 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:20.989761 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:20.989767 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:20.989770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:20.989939 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:20.990409 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:20.990412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:20.990414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:20.990973 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:20.991130 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:20.991701 (+ 571us) server_negotiation.cc:299] Negotiation successful
0926 15:08:20.991962 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1607,"spinlock_wait_cycles":23936,"thread_start_us":1360,"threads_started":1}
I20260926 15:08:20.993160 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:20.973563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39155 (local address 127.0.0.1:39808)
0926 15:08:20.974449 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:20.974501 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0926 15:08:20.976891 (+ 2390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:08:20.977771 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:08:20.977777 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:08:20.978475 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:08:20.978829 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:20.978847 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:20.980380 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:20.980384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:20.981555 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:20.981568 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:20.982527 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:20.982534 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:20.982698 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:20.983313 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:08:20.983344 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:08:20.985792 (+ 2448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:08:20.988986 (+ 3194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:20.989000 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:20.989013 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:20.989627 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:20.990059 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:20.990064 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:20.990068 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:20.990305 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:20.991103 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:08:20.991109 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:08:20.991328 (+ 219us) client_negotiation.cc:769] Sending connection context
0926 15:08:20.991543 (+ 215us) client_negotiation.cc:240] Negotiation successful
0926 15:08:20.992104 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":134,"thread_start_us":350,"threads_started":1}
I20260926 15:08:20.996805 32585 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:08:20.997931 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:20.997551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:42053 (local address 127.0.0.1:59998)
0926 15:08:20.997708 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:20.997814 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:42053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260926 15:08:20.998204 32534 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.123.125:42053: connect: Connection refused (error 111)
I20260926 15:08:20.999748 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:20.999359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:42053 (local address 127.0.0.1:60002)
0926 15:08:20.999525 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:20.999597 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:42053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260926 15:08:20.999934 32585 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.125:42053: Network error: Client connection negotiation failed: client connection to 127.9.123.125:42053: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.125.keytab.
Entry for principal kudu/127.9.123.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.125.keytab.
I20260926 15:08:21.024448 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.125:42053
--webserver_interface=127.9.123.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.125.keytab
--principal=kudu/127.9.123.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:43093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:39155,127.9.123.125:42053,127.9.123.124:36169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:08:21.049304 32585 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } attempt: 1
I20260926 15:08:21.050437 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.049984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:42053 (local address 127.0.0.1:60018)
0926 15:08:21.050185 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.050290 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:42053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260926 15:08:21.051496 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.051114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:42053 (local address 127.0.0.1:60032)
0926 15:08:21.051264 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.051358 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:42053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260926 15:08:21.051687 32585 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.125:42053: Network error: Client connection negotiation failed: client connection to 127.9.123.125:42053: connect: Connection refused (error 111)
I20260926 15:08:21.119937 32585 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } attempt: 2
I20260926 15:08:21.121068 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.120575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:42053 (local address 127.0.0.1:60038)
0926 15:08:21.120823 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.120925 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:42053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260926 15:08:21.122284 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.121832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:42053 (local address 127.0.0.1:60040)
0926 15:08:21.122045 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.122140 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:42053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260926 15:08:21.122509 32585 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.125:42053: Network error: Client connection negotiation failed: client connection to 127.9.123.125:42053: connect: Connection refused (error 111)
I20260926 15:08:21.213798 32585 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } attempt: 3
I20260926 15:08:21.214900 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.214470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:42053 (local address 127.0.0.1:60052)
0926 15:08:21.214669 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.214763 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:42053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260926 15:08:21.216033 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.215682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:42053 (local address 127.0.0.1:60066)
0926 15:08:21.215824 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.215897 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:42053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260926 15:08:21.216264 32585 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.125:42053: Network error: Client connection negotiation failed: client connection to 127.9.123.125:42053: connect: Connection refused (error 111)
W20260926 15:08:21.354116 32592 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 15:08:21.354450 32592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:08:21.354523 32592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:08:21.354604 32592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260926 15:08:21.359493 32585 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } attempt: 4
I20260926 15:08:21.360714 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.360171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:42053 (local address 127.0.0.1:60072)
0926 15:08:21.360477 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.360568 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:42053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168}
I20260926 15:08:21.361735 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.361372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:42053 (local address 127.0.0.1:60076)
0926 15:08:21.361500 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.361594 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.125:42053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260926 15:08:21.361905 32585 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.125:42053: Network error: Client connection negotiation failed: client connection to 127.9.123.125:42053: connect: Connection refused (error 111)
W20260926 15:08:21.362562 32592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:08:21.362660 32592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:08:21.362777 32592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:08:21.362834 32592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:08:21.362887 32592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:08:21.372975 32592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:43093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:39155,127.9.123.125:42053,127.9.123.124:36169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.125.keytab
--principal=kudu/127.9.123.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.125:42053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.123.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:08:21.374333 32592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:08:21.375639 32592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:08:21.387718 32598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:21.387746 32597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:21.388150 32600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:08:21.388553 32592 server_base.cc:1034] running on GCE node
I20260926 15:08:21.389436 32592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:08:21.390961 32592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:08:21.392263 32592 hybrid_clock.cc:648] HybridClock initialized: now 1790435301392207 us; error 63 us; skew 500 ppm
I20260926 15:08:21.392625 32592 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:08:21 dist-test-slave-7153 krb5kdc[32505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435301, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:08:21.395907 32592 init.cc:377] Logged in from keytab as kudu/127.9.123.125@KRBTEST.COM (short username kudu)
I20260926 15:08:21.397472 32592 webserver.cc:460] Webserver started at http://127.9.123.125:41527/ using document root <none> and password file <none>
I20260926 15:08:21.398178 32592 fs_manager.cc:359] Metadata directory not provided
I20260926 15:08:21.398268 32592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:08:21.398532 32592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:08:21.401621 32592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-1/data/instance:
uuid: "16a0833d0b5a46a3b6db5a4ee69e66bc"
format_stamp: "Formatted at 2026-09-26 15:08:21 on dist-test-slave-7153"
server_key: "4874e67aa79de77d08e8e1119ad7a0bb"
server_key_iv: "a24f3bbcf4409f0c89fc5f8831b54a35"
server_key_version: "encryptionkey@0"
I20260926 15:08:21.402361 32592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-1/wal/instance:
uuid: "16a0833d0b5a46a3b6db5a4ee69e66bc"
format_stamp: "Formatted at 2026-09-26 15:08:21 on dist-test-slave-7153"
server_key: "4874e67aa79de77d08e8e1119ad7a0bb"
server_key_iv: "a24f3bbcf4409f0c89fc5f8831b54a35"
server_key_version: "encryptionkey@0"
I20260926 15:08:21.407582 32592 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 15:08:21.411072 32607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:08:21.412539 32592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260926 15:08:21.412696 32592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-1/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-1/wal
uuid: "16a0833d0b5a46a3b6db5a4ee69e66bc"
format_stamp: "Formatted at 2026-09-26 15:08:21 on dist-test-slave-7153"
server_key: "4874e67aa79de77d08e8e1119ad7a0bb"
server_key_iv: "a24f3bbcf4409f0c89fc5f8831b54a35"
server_key_version: "encryptionkey@0"
I20260926 15:08:21.412902 32592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:08:21.443337 32592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:08:21.447611 32592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:08:21.447851 32592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:08:21.465396 32592 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.125:42053
I20260926 15:08:21.465417 32658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.125:42053 every 8 connection(s)
I20260926 15:08:21.466753 32592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-1/data/info.pb
I20260926 15:08:21.470202 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 32592
I20260926 15:08:21.470373 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-1/wal/instance
I20260926 15:08:21.470563 32659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:08:21.470836 9709 external_mini_cluster.cc:1428] Setting key 625ecc508db7cd5722c2cb3bb0fd8a91
I20260926 15:08:21.477231 32659 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39155 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:08:21 dist-test-slave-7153 krb5kdc[32505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435301, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.125@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.123.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.123.124@KRBTEST.COM" created.
I20260926 15:08:21.498970 32588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.480166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39818 (local address 127.9.123.126:39155)
0926 15:08:21.480385 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:21.480391 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:21.481839 (+ 1448us) server_negotiation.cc:407] Connection header received
0926 15:08:21.483818 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:21.483825 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:21.483915 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:21.484038 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:21.485778 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:21.486683 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:21.488331 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:21.488962 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:21.489091 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:21.492444 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:21.492449 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:21.492453 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:21.492490 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:21.495430 (+ 2940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:21.496470 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:21.496475 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:21.496477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:21.496607 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:21.497075 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:21.497078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:21.497080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:21.497433 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:21.497551 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:21.498498 (+ 947us) server_negotiation.cc:299] Negotiation successful
0926 15:08:21.498734 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260926 15:08:21.500330 32662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.480784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39155 (local address 127.0.0.1:39818)
0926 15:08:21.481638 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.481703 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0926 15:08:21.483515 (+ 1812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:08:21.484316 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:08:21.484326 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:08:21.484907 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:08:21.485592 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:21.485613 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:21.486859 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:21.486865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:21.488166 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:21.488179 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:21.489203 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:21.489209 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:21.489417 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:21.490043 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:08:21.490067 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:08:21.492200 (+ 2133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:08:21.495624 (+ 3424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:21.495635 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:21.495665 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:21.496330 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:21.496716 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:21.496720 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:21.496724 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:21.496975 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:21.497561 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:08:21.497570 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:08:21.498054 (+ 484us) client_negotiation.cc:769] Sending connection context
0926 15:08:21.498356 (+ 302us) client_negotiation.cc:240] Negotiation successful
0926 15:08:21.498924 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":376,"threads_started":1}
I20260926 15:08:21.502280 32659 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:08:21 dist-test-slave-7153 krb5kdc[32505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435301, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.125@KRBTEST.COM for kudu/127.9.123.125@KRBTEST.COM
I20260926 15:08:21.517503 32662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.503061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:42053 (local address 127.0.0.1:60092)
0926 15:08:21.503246 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.503262 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0926 15:08:21.503347 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:08:21.504442 (+ 1095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:08:21.504460 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:08:21.504928 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:08:21.505068 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:21.505076 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:21.506436 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:21.506441 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:21.507277 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:21.507288 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:21.508091 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:21.508096 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:21.508259 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:21.508899 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:08:21.508916 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:08:21.511218 (+ 2302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:08:21.514675 (+ 3457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:21.514680 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:21.514700 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:21.515077 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:21.515457 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:21.515463 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:21.515466 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:21.515598 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:21.516461 (+ 863us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:08:21.516465 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:08:21.516615 (+ 150us) client_negotiation.cc:769] Sending connection context
0926 15:08:21.516758 (+ 143us) client_negotiation.cc:240] Negotiation successful
0926 15:08:21.517064 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260926 15:08:21.517629 32663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.503331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60092 (local address 127.9.123.125:42053)
0926 15:08:21.503977 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:21.503983 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:21.504009 (+ 26us) server_negotiation.cc:407] Connection header received
0926 15:08:21.504057 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:21.504063 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:21.504280 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:21.504440 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:21.505257 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:21.506308 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:21.507400 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:21.507935 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:21.508044 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:21.511483 (+ 3439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:21.511503 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:21.511525 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:21.511563 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:21.514519 (+ 2956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:21.515221 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:21.515228 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:21.515231 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:21.515351 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:21.515714 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:21.515719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:21.515723 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:21.516328 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:21.516564 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:21.516920 (+ 356us) server_negotiation.cc:299] Negotiation successful
0926 15:08:21.517279 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":410,"threads_started":1}
I20260926 15:08:21.520277 32659 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:08:21.521582 32662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.521002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:36169 (local address 127.0.0.1:45958)
0926 15:08:21.521230 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.521400 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260926 15:08:21.521844 32608 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
I20260926 15:08:21.523452 32662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.523059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:36169 (local address 127.0.0.1:45964)
0926 15:08:21.523226 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.523326 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260926 15:08:21.523689 32659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:36169: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.123.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.124.keytab.
Entry for principal kudu/127.9.123.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.124.keytab.
I20260926 15:08:21.532655 9709 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskR962dp/build/asan/bin/kudu
/tmp/dist-test-taskR962dp/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.124:36169
--webserver_interface=127.9.123.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.124.keytab
--principal=kudu/127.9.123.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.123.84:43093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.123.126:39155,127.9.123.125:42053,127.9.123.124:36169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260926 15:08:21.573041 32659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } attempt: 1
I20260926 15:08:21.574255 32662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.573801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:36169 (local address 127.0.0.1:45966)
0926 15:08:21.573993 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.574103 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260926 15:08:21.575328 32662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.574923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:36169 (local address 127.0.0.1:45978)
0926 15:08:21.575117 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.575207 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260926 15:08:21.575559 32659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:36169: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
I20260926 15:08:21.643801 32659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } attempt: 2
I20260926 15:08:21.644901 32662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.644451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:36169 (local address 127.0.0.1:45988)
0926 15:08:21.644654 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.644746 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260926 15:08:21.646090 32662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.645686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:36169 (local address 127.0.0.1:45992)
0926 15:08:21.645867 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.645948 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260926 15:08:21.646360 32659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:36169: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
I20260926 15:08:21.661206 32585 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } attempt: 5
Sep 26 15:08:21 dist-test-slave-7153 krb5kdc[32505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435300, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for kudu/127.9.123.125@KRBTEST.COM
I20260926 15:08:21.675724 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.661931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:42053 (local address 127.0.0.1:60094)
0926 15:08:21.662147 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.662171 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0926 15:08:21.662290 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:08:21.662648 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:08:21.662651 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:08:21.663037 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:08:21.663213 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:21.663222 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:21.664599 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:21.664603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:21.665555 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:21.665569 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:21.666492 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:21.666496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:21.666683 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:21.667358 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:08:21.667377 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:08:21.669579 (+ 2202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:08:21.673284 (+ 3705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:21.673288 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:21.673291 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:21.673650 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:21.674088 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:21.674091 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:21.674095 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:21.674239 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:21.674952 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:08:21.674956 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:08:21.675104 (+ 148us) client_negotiation.cc:769] Sending connection context
0926 15:08:21.675230 (+ 126us) client_negotiation.cc:240] Negotiation successful
0926 15:08:21.675479 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260926 15:08:21.675922 32663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.662044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60094 (local address 127.9.123.125:42053)
0926 15:08:21.662293 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:21.662300 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:21.662326 (+ 26us) server_negotiation.cc:407] Connection header received
0926 15:08:21.662395 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:21.662401 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:21.662508 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:21.662622 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:21.663364 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:21.664434 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:21.665717 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:21.666337 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:21.666459 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:21.669794 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:21.669800 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:21.669805 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:21.669844 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:21.673144 (+ 3300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:21.673777 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:21.673784 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:21.673787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:21.673944 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:21.674357 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:21.674363 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:21.674366 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:21.674805 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:21.674966 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:21.675354 (+ 388us) server_negotiation.cc:299] Negotiation successful
0926 15:08:21.675602 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260926 15:08:21.677016 32585 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } has no permanent_uuid. Determining permanent_uuid...
I20260926 15:08:21.678122 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.677665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:36169 (local address 127.0.0.1:46002)
0926 15:08:21.677849 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.677945 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260926 15:08:21.679119 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.678724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:36169 (local address 127.0.0.1:46016)
0926 15:08:21.678879 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.678986 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 15:08:21.679353 32585 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:36169: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
I20260926 15:08:21.730604 32585 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } attempt: 1
I20260926 15:08:21.731951 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.731315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:36169 (local address 127.0.0.1:46032)
0926 15:08:21.731577 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.731718 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260926 15:08:21.733080 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.732700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:36169 (local address 127.0.0.1:46038)
0926 15:08:21.732853 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.732943 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260926 15:08:21.733352 32585 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:36169: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
I20260926 15:08:21.737578 32659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } attempt: 3
I20260926 15:08:21.738564 32662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.738106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:36169 (local address 127.0.0.1:46044)
0926 15:08:21.738331 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.738415 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260926 15:08:21.739578 32662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.739133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:36169 (local address 127.0.0.1:46048)
0926 15:08:21.739386 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.739463 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
W20260926 15:08:21.739768 32659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:36169: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
I20260926 15:08:21.801571 32585 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } attempt: 2
I20260926 15:08:21.802846 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.802298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:36169 (local address 127.0.0.1:46060)
0926 15:08:21.802570 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.802667 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260926 15:08:21.803995 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.803627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:36169 (local address 127.0.0.1:46072)
0926 15:08:21.803785 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.803860 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260926 15:08:21.804231 32585 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:36169: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
W20260926 15:08:21.863974 32666 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260926 15:08:21.864359 32666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260926 15:08:21.864459 32666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260926 15:08:21.864573 32666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260926 15:08:21.872036 32666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260926 15:08:21.872155 32666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260926 15:08:21.872249 32666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260926 15:08:21.872316 32666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260926 15:08:21.872404 32666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260926 15:08:21.882741 32666 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.123.84:43093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.123.126:39155,127.9.123.125:42053,127.9.123.124:36169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/krb5kdc/kudu_127.9.123.124.keytab
--principal=kudu/127.9.123.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.123.124:36169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.123.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 26 Sep 2026 14:12:06 UTC on ccycloud.quasar-ygbmzu.root.comops.site
build id 4674337
ASAN enabled
I20260926 15:08:21.883004 32659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } attempt: 4
I20260926 15:08:21.884183 32662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.883700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:36169 (local address 127.0.0.1:46074)
0926 15:08:21.883931 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.884018 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260926 15:08:21.884157 32666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260926 15:08:21.885257 32662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.884843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:36169 (local address 127.0.0.1:46078)
0926 15:08:21.884993 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.885063 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260926 15:08:21.885447 32659 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:36169: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
I20260926 15:08:21.885705 32666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260926 15:08:21.898348 32671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260926 15:08:21.898330 32674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:08:21.898939 32666 server_base.cc:1034] running on GCE node
W20260926 15:08:21.899003 32672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260926 15:08:21.899950 32666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260926 15:08:21.901585 32666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260926 15:08:21.902856 32666 hybrid_clock.cc:648] HybridClock initialized: now 1790435301902784 us; error 46 us; skew 500 ppm
I20260926 15:08:21.903250 32666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 26 15:08:21 dist-test-slave-7153 krb5kdc[32505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435301, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260926 15:08:21.906633 32666 init.cc:377] Logged in from keytab as kudu/127.9.123.124@KRBTEST.COM (short username kudu)
I20260926 15:08:21.908380 32666 webserver.cc:460] Webserver started at http://127.9.123.124:39871/ using document root <none> and password file <none>
I20260926 15:08:21.909142 32666 fs_manager.cc:359] Metadata directory not provided
I20260926 15:08:21.909252 32666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260926 15:08:21.909541 32666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260926 15:08:21.910449 32585 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } attempt: 3
I20260926 15:08:21.911490 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.911024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:36169 (local address 127.0.0.1:46084)
0926 15:08:21.911249 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.911332 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260926 15:08:21.912513 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.912171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:36169 (local address 127.0.0.1:46100)
0926 15:08:21.912334 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:21.912408 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260926 15:08:21.912724 32585 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.123.124:36169: Network error: Client connection negotiation failed: client connection to 127.9.123.124:36169: connect: Connection refused (error 111)
I20260926 15:08:21.912914 32666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-2/data/instance:
uuid: "24ef1a30fdec4655a65300e0e6bf2619"
format_stamp: "Formatted at 2026-09-26 15:08:21 on dist-test-slave-7153"
server_key: "878abaec9024afd7f1050e9dce4f8c57"
server_key_iv: "70958594df16e8a07760041ad53cf728"
server_key_version: "encryptionkey@0"
I20260926 15:08:21.913748 32666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-2/wal/instance:
uuid: "24ef1a30fdec4655a65300e0e6bf2619"
format_stamp: "Formatted at 2026-09-26 15:08:21 on dist-test-slave-7153"
server_key: "878abaec9024afd7f1050e9dce4f8c57"
server_key_iv: "70958594df16e8a07760041ad53cf728"
server_key_version: "encryptionkey@0"
I20260926 15:08:21.919301 32666 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260926 15:08:21.922926 32681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260926 15:08:21.924420 32666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260926 15:08:21.924579 32666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-2/data,/tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-2/wal
uuid: "24ef1a30fdec4655a65300e0e6bf2619"
format_stamp: "Formatted at 2026-09-26 15:08:21 on dist-test-slave-7153"
server_key: "878abaec9024afd7f1050e9dce4f8c57"
server_key_iv: "70958594df16e8a07760041ad53cf728"
server_key_version: "encryptionkey@0"
I20260926 15:08:21.924803 32666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260926 15:08:21.964772 32666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260926 15:08:21.969578 32666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260926 15:08:21.969810 32666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260926 15:08:21.987093 32666 rpc_server.cc:307] RPC server started. Bound to: 127.9.123.124:36169
I20260926 15:08:21.987107 32732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.123.124:36169 every 8 connection(s)
I20260926 15:08:21.988396 32666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-2/data/info.pb
I20260926 15:08:21.988911 9709 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskR962dp/build/asan/bin/kudu as pid 32666
I20260926 15:08:21.989077 9709 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskR962dp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790434939095847-9709-0/minicluster-data/master-2/wal/instance
I20260926 15:08:21.989656 9709 external_mini_cluster.cc:1428] Setting key ada090c6ba0e85fddb2f24b7e465a67d
I20260926 15:08:21.992630 32733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260926 15:08:21.999785 32733 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39155 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:08:21 dist-test-slave-7153 krb5kdc[32505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.124@KRBTEST.COM
Sep 26 15:08:22 dist-test-slave-7153 krb5kdc[32505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435301, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.124@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
I20260926 15:08:22.012318 32736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:21.991417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46104 (local address 127.9.123.124:36169)
0926 15:08:21.992189 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:21.992201 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:21.992252 (+ 51us) server_negotiation.cc:407] Connection header received
0926 15:08:21.993976 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:21.994000 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:21.994416 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:21.994784 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:21.996239 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:21.997672 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:21.998986 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:21.999725 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:21.999899 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:22.004646 (+ 4747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:22.004654 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:22.004664 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:22.004701 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:22.007858 (+ 3157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:22.008583 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:22.008592 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:22.008601 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:22.008789 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:22.009196 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:22.009201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:22.009204 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:22.009806 (+ 602us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:22.010058 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:22.010621 (+ 563us) server_negotiation.cc:299] Negotiation successful
0926 15:08:22.011090 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":326,"threads_started":1}
I20260926 15:08:22.018079 32738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:22.002782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.126:39155 (local address 127.0.0.1:39822)
0926 15:08:22.003616 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:22.003664 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0926 15:08:22.003822 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:08:22.004192 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:08:22.004200 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:08:22.004840 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:08:22.005178 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:22.005190 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.006685 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.006689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:22.007806 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:22.007818 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.008585 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.008590 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:22.008733 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:22.009364 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:08:22.009381 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:08:22.011825 (+ 2444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:08:22.015075 (+ 3250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:22.015088 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:22.015094 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:22.015714 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:22.016101 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:22.016106 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:22.016109 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:22.016281 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:22.016897 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:08:22.016906 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:08:22.017180 (+ 274us) client_negotiation.cc:769] Sending connection context
0926 15:08:22.017395 (+ 215us) client_negotiation.cc:240] Negotiation successful
0926 15:08:22.017769 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":481,"threads_started":1}
I20260926 15:08:22.018570 32737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:22.002792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39822 (local address 127.9.123.126:39155)
0926 15:08:22.003277 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:22.003283 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:22.003751 (+ 468us) server_negotiation.cc:407] Connection header received
0926 15:08:22.003938 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:22.003942 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:22.004007 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:22.004096 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:22.005351 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.006519 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.007948 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.008391 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.008487 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:22.012038 (+ 3551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:22.012045 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:22.012050 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:22.012095 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:22.014939 (+ 2844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:22.015823 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:22.015829 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:22.015832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:22.016004 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:22.016395 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:22.016398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:22.016419 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:22.016752 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:22.016881 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:22.017521 (+ 640us) server_negotiation.cc:299] Negotiation successful
0926 15:08:22.017770 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":227,"threads_started":1}
I20260926 15:08:22.020121 32733 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:08:22 dist-test-slave-7153 krb5kdc[32505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.126@KRBTEST.COM
Sep 26 15:08:22 dist-test-slave-7153 krb5kdc[32505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435301, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.124@KRBTEST.COM for kudu/127.9.123.125@KRBTEST.COM
I20260926 15:08:22.032780 32739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:22.015304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39838 (local address 127.9.123.126:39155)
0926 15:08:22.015913 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:22.015919 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:22.015940 (+ 21us) server_negotiation.cc:407] Connection header received
0926 15:08:22.015984 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:22.015988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:22.016052 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:22.016145 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:22.017738 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.019003 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.020716 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.021348 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.021480 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:22.026429 (+ 4949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:22.026439 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:22.026444 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:22.026493 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:22.029708 (+ 3215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:22.030572 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:22.030577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:22.030579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:22.030717 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:22.031140 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:22.031145 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:22.031149 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:22.031545 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:22.031843 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:22.032209 (+ 366us) server_negotiation.cc:299] Negotiation successful
0926 15:08:22.032435 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":264,"threads_started":1}
I20260926 15:08:22.036671 32663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:22.020951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60110 (local address 127.9.123.125:42053)
0926 15:08:22.021166 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:22.021172 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:22.021191 (+ 19us) server_negotiation.cc:407] Connection header received
0926 15:08:22.021307 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:22.021313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:22.021393 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:22.021489 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:22.022448 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.023143 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.024847 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.025589 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.025735 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:22.029516 (+ 3781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:22.029522 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:22.029526 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:22.029564 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:22.033678 (+ 4114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:22.034485 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:22.034491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:22.034495 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:22.034709 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:22.035151 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:22.035158 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:22.035162 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:22.035477 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:22.035687 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:22.036221 (+ 534us) server_negotiation.cc:299] Negotiation successful
0926 15:08:22.036392 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260926 15:08:22.036798 32738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:22.020879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.125:42053 (local address 127.0.0.1:60110)
0926 15:08:22.021051 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:22.021070 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0926 15:08:22.021204 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:08:22.021557 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:08:22.021562 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:08:22.022087 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:08:22.022273 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:22.022285 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.023306 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.023313 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:22.024688 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:22.024702 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.025779 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.025784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:22.025927 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:22.026511 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:08:22.026534 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:08:22.029131 (+ 2597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:08:22.033836 (+ 4705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:22.033842 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:22.033846 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:22.034353 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:22.034847 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:22.034852 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:22.034856 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:22.035026 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:22.035777 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:08:22.035782 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:08:22.035975 (+ 193us) client_negotiation.cc:769] Sending connection context
0926 15:08:22.036133 (+ 158us) client_negotiation.cc:240] Negotiation successful
0926 15:08:22.036447 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260926 15:08:22.037904 32733 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } has no permanent_uuid. Determining permanent_uuid...
Sep 26 15:08:22 dist-test-slave-7153 krb5kdc[32505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.123.125@KRBTEST.COM
Sep 26 15:08:22 dist-test-slave-7153 krb5kdc[32505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435301, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.124@KRBTEST.COM for kudu/127.9.123.124@KRBTEST.COM
I20260926 15:08:22.051806 32740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:22.034710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60118 (local address 127.9.123.125:42053)
0926 15:08:22.035320 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:22.035324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:22.035342 (+ 18us) server_negotiation.cc:407] Connection header received
0926 15:08:22.035381 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:22.035384 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:22.035438 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:22.035527 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:22.037078 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.037991 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.039615 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.040254 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.040346 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:22.045200 (+ 4854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:22.045211 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:22.045216 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:22.045272 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:22.048792 (+ 3520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:22.049616 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:22.049622 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:22.049625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:22.049794 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:22.050274 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:22.050284 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:22.050287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:22.050686 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:22.050924 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:22.051221 (+ 297us) server_negotiation.cc:299] Negotiation successful
0926 15:08:22.051444 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":276,"threads_started":1}
I20260926 15:08:22.052342 32736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:22.038903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46108 (local address 127.9.123.124:36169)
0926 15:08:22.039063 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:22.039069 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:22.039102 (+ 33us) server_negotiation.cc:407] Connection header received
0926 15:08:22.039148 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:22.039153 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:22.039222 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:22.039317 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:22.040143 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.041039 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.042223 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.042877 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.043009 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:22.046609 (+ 3600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:22.046615 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:22.046620 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:22.046659 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:22.049567 (+ 2908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:22.050301 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:22.050306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:22.050309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:22.050456 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:22.050868 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:22.050873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:22.050877 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:22.051195 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:22.051332 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:22.051746 (+ 414us) server_negotiation.cc:299] Negotiation successful
0926 15:08:22.052012 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":53}
I20260926 15:08:22.052778 32738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:22.038693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:36169 (local address 127.0.0.1:46108)
0926 15:08:22.038896 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:22.038914 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:08:22.039012 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:08:22.039345 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:08:22.039348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:08:22.039784 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:08:22.039956 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:22.039967 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.041217 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.041222 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:22.042077 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:22.042088 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.043075 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.043080 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:22.043258 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:22.043792 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:08:22.043809 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:08:22.046396 (+ 2587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:08:22.049715 (+ 3319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:22.049719 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:22.049722 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:22.050179 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:22.050584 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:22.050588 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:22.050590 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:22.050734 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:22.051328 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:08:22.051333 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:08:22.051491 (+ 158us) client_negotiation.cc:769] Sending connection context
0926 15:08:22.051654 (+ 163us) client_negotiation.cc:240] Negotiation successful
0926 15:08:22.052528 (+ 874us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":526}
I20260926 15:08:22.056958 9709 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260926 15:08:22.058396 32733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24ef1a30fdec4655a65300e0e6bf2619: Bootstrap starting.
I20260926 15:08:22.062011 32733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24ef1a30fdec4655a65300e0e6bf2619: Neither blocks nor log segments found. Creating new log.
I20260926 15:08:22.060025 32560 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "14e19504a2ad4639a3c9f3ce785e8176"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39838
I20260926 15:08:22.063117 32733 log.cc:824] T 00000000000000000000000000000000 P 24ef1a30fdec4655a65300e0e6bf2619: Log is configured to *not* fsync() on all Append() calls
I20260926 15:08:22.065961 32733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24ef1a30fdec4655a65300e0e6bf2619: No bootstrap required, opened a new log
I20260926 15:08:22.073290 32733 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24ef1a30fdec4655a65300e0e6bf2619 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14e19504a2ad4639a3c9f3ce785e8176" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39155 } } peers { permanent_uuid: "16a0833d0b5a46a3b6db5a4ee69e66bc" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } } peers { permanent_uuid: "24ef1a30fdec4655a65300e0e6bf2619" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } }
I20260926 15:08:22.073659 32733 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24ef1a30fdec4655a65300e0e6bf2619 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24ef1a30fdec4655a65300e0e6bf2619, State: Initialized, Role: FOLLOWER
I20260926 15:08:22.074134 32733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24ef1a30fdec4655a65300e0e6bf2619 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14e19504a2ad4639a3c9f3ce785e8176" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39155 } } peers { permanent_uuid: "16a0833d0b5a46a3b6db5a4ee69e66bc" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } } peers { permanent_uuid: "24ef1a30fdec4655a65300e0e6bf2619" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } }
I20260926 15:08:22.075139 32742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24ef1a30fdec4655a65300e0e6bf2619 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14e19504a2ad4639a3c9f3ce785e8176" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39155 } } peers { permanent_uuid: "16a0833d0b5a46a3b6db5a4ee69e66bc" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } } peers { permanent_uuid: "24ef1a30fdec4655a65300e0e6bf2619" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } } }
I20260926 15:08:22.075553 32742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24ef1a30fdec4655a65300e0e6bf2619 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:08:22.076213 32733 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24ef1a30fdec4655a65300e0e6bf2619 [sys.catalog]: configured and running, proceeding with master startup.
W20260926 15:08:22.086259 32753 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 24ef1a30fdec4655a65300e0e6bf2619: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260926 15:08:22.086426 32753 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260926 15:08:22.089920 32585 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } attempt: 4
I20260926 15:08:22.083663 32666 master_runner.cc:186] Error getting master registration for 127.9.123.126:39155: OK, instance_id { permanent_uuid: "14e19504a2ad4639a3c9f3ce785e8176" instance_seqno: 1790435300952051 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 15:08:22.094779 32666 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 26 15:08:22 dist-test-slave-7153 krb5kdc[32505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435300, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.126@KRBTEST.COM for kudu/127.9.123.124@KRBTEST.COM
I20260926 15:08:22.107466 32587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:22.090669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:36169 (local address 127.0.0.1:46112)
0926 15:08:22.090953 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:22.090974 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0926 15:08:22.091116 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:08:22.091537 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:08:22.091542 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:08:22.092038 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:08:22.092270 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:22.092285 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.093822 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.093828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:22.095190 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:22.095211 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.096063 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.096070 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:22.096304 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:22.096989 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:08:22.097014 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:08:22.099898 (+ 2884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:08:22.104220 (+ 4322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:22.104227 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:22.104232 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:22.104842 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:22.105341 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:22.105348 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:22.105352 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:22.105537 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:22.106334 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:08:22.106339 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:08:22.106544 (+ 205us) client_negotiation.cc:769] Sending connection context
0926 15:08:22.106721 (+ 177us) client_negotiation.cc:240] Negotiation successful
0926 15:08:22.107046 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164}
I20260926 15:08:22.107474 32736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:22.090771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46112 (local address 127.9.123.124:36169)
0926 15:08:22.090987 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:22.090993 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:22.091053 (+ 60us) server_negotiation.cc:407] Connection header received
0926 15:08:22.091264 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:22.091270 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:22.091360 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:22.091496 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:22.092462 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.093651 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.095376 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.095886 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.096005 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:22.100153 (+ 4148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:22.100164 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:22.100169 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:22.100215 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:22.104008 (+ 3793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:22.104995 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:22.105000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:22.105004 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:22.105214 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:22.105648 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:22.105653 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:22.105659 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:22.106179 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:22.106350 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:22.106843 (+ 493us) server_negotiation.cc:299] Negotiation successful
0926 15:08:22.107108 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 15:08:22.113813 32585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176: Bootstrap starting.
I20260926 15:08:22.117123 32585 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176: Neither blocks nor log segments found. Creating new log.
I20260926 15:08:22.118260 32585 log.cc:824] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176: Log is configured to *not* fsync() on all Append() calls
I20260926 15:08:22.121265 32585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176: No bootstrap required, opened a new log
I20260926 15:08:22.126428 32585 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14e19504a2ad4639a3c9f3ce785e8176" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39155 } } peers { permanent_uuid: "16a0833d0b5a46a3b6db5a4ee69e66bc" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } } peers { permanent_uuid: "24ef1a30fdec4655a65300e0e6bf2619" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } }
I20260926 15:08:22.126781 32585 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14e19504a2ad4639a3c9f3ce785e8176, State: Initialized, Role: FOLLOWER
I20260926 15:08:22.127265 32585 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14e19504a2ad4639a3c9f3ce785e8176" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39155 } } peers { permanent_uuid: "16a0833d0b5a46a3b6db5a4ee69e66bc" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } } peers { permanent_uuid: "24ef1a30fdec4655a65300e0e6bf2619" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } }
I20260926 15:08:22.128504 32756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14e19504a2ad4639a3c9f3ce785e8176" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39155 } } peers { permanent_uuid: "16a0833d0b5a46a3b6db5a4ee69e66bc" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } } peers { permanent_uuid: "24ef1a30fdec4655a65300e0e6bf2619" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } } }
I20260926 15:08:22.129397 32585 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176 [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:08:22.129488 32560 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260926 15:08:22.129694 32560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:08:22.133383 32560 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14e19504a2ad4639a3c9f3ce785e8176" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39155 } } peers { permanent_uuid: "16a0833d0b5a46a3b6db5a4ee69e66bc" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } } peers { permanent_uuid: "24ef1a30fdec4655a65300e0e6bf2619" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } }
I20260926 15:08:22.135430 32560 leader_election.cc:288] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176 [CANDIDATE]: Term 1 election: Requested vote from peers 16a0833d0b5a46a3b6db5a4ee69e66bc (127.9.123.125:42053), 24ef1a30fdec4655a65300e0e6bf2619 (127.9.123.124:36169)
I20260926 15:08:22.135769 32756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:08:22.136606 32708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "14e19504a2ad4639a3c9f3ce785e8176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "24ef1a30fdec4655a65300e0e6bf2619"
I20260926 15:08:22.137168 32708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24ef1a30fdec4655a65300e0e6bf2619 [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:08:22.140656 32708 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 24ef1a30fdec4655a65300e0e6bf2619 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14e19504a2ad4639a3c9f3ce785e8176 in term 1.
I20260926 15:08:22.141398 32534 leader_election.cc:302] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14e19504a2ad4639a3c9f3ce785e8176, 24ef1a30fdec4655a65300e0e6bf2619; no voters:
I20260926 15:08:22.142016 32756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176 [term 1 FOLLOWER]: Leader election won for term 1
I20260926 15:08:22.146646 300 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260926 15:08:22.149367 32756 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176 [term 1 LEADER]: Becoming Leader. State: Replica: 14e19504a2ad4639a3c9f3ce785e8176, State: Running, Role: LEADER
I20260926 15:08:22.150060 32756 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14e19504a2ad4639a3c9f3ce785e8176" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39155 } } peers { permanent_uuid: "16a0833d0b5a46a3b6db5a4ee69e66bc" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } } peers { permanent_uuid: "24ef1a30fdec4655a65300e0e6bf2619" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } }
I20260926 15:08:22.154847 301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14e19504a2ad4639a3c9f3ce785e8176. Latest consensus state: current_term: 1 leader_uuid: "14e19504a2ad4639a3c9f3ce785e8176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14e19504a2ad4639a3c9f3ce785e8176" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39155 } } peers { permanent_uuid: "16a0833d0b5a46a3b6db5a4ee69e66bc" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } } peers { permanent_uuid: "24ef1a30fdec4655a65300e0e6bf2619" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } } }
I20260926 15:08:22.155309 301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176 [sys.catalog]: This master's current role is: LEADER
I20260926 15:08:22.158218 305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260926 15:08:22.172631 305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260926 15:08:22.157357 32518 master_runner.cc:186] Error getting master registration for 127.9.123.125:42053: OK, instance_id { permanent_uuid: "16a0833d0b5a46a3b6db5a4ee69e66bc" instance_seqno: 1790435301460410 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260926 15:08:22.178478 32518 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260926 15:08:22.184638 32659 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } attempt: 5
W20260926 15:08:22.192615 32539 tablet.cc:2369] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:08:22.205297 32708 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 24ef1a30fdec4655a65300e0e6bf2619 [term 1 FOLLOWER]: Refusing update from remote peer 14e19504a2ad4639a3c9f3ce785e8176: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:08:22.206537 301 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24ef1a30fdec4655a65300e0e6bf2619" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:08:22.207320 32739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:22.168673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39854 (local address 127.9.123.126:39155)
0926 15:08:22.169339 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:22.169345 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:22.177519 (+ 8174us) server_negotiation.cc:407] Connection header received
0926 15:08:22.177944 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:22.177950 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:22.178039 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:22.178109 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:22.184981 (+ 6872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.186141 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.189914 (+ 3773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.190603 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.190708 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:22.197872 (+ 7164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:22.197879 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:22.197885 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:22.197932 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:22.202082 (+ 4150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:22.203532 (+ 1450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:22.203539 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:22.203543 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:22.203698 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:22.204461 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:22.204467 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:22.204471 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:22.204864 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:22.205150 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:22.206197 (+ 1047us) server_negotiation.cc:299] Negotiation successful
0926 15:08:22.206466 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
Sep 26 15:08:22 dist-test-slave-7153 krb5kdc[32505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790435301, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.123.125@KRBTEST.COM for kudu/127.9.123.124@KRBTEST.COM
W20260926 15:08:22.221927 32687 tablet.cc:2369] T 00000000000000000000000000000000 P 24ef1a30fdec4655a65300e0e6bf2619: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260926 15:08:22.223433 32740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:22.170246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60132 (local address 127.9.123.125:42053)
0926 15:08:22.170423 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:22.170430 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:22.182877 (+ 12447us) server_negotiation.cc:407] Connection header received
0926 15:08:22.184211 (+ 1334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:22.184217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:22.184334 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:22.184407 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:22.188537 (+ 4130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.189651 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.198201 (+ 8550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.198955 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.199066 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:22.211181 (+ 12115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:22.211188 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:22.211193 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:22.211240 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:22.217226 (+ 5986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:22.219047 (+ 1821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:22.219054 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:22.219058 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:22.219228 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:22.219671 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:22.219677 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:22.219680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:22.220063 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:22.220330 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:22.221564 (+ 1234us) server_negotiation.cc:299] Negotiation successful
0926 15:08:22.223035 (+ 1471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260926 15:08:22.245079 32662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:22.185204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.123.124:36169 (local address 127.0.0.1:46140)
0926 15:08:22.185541 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0926 15:08:22.185559 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0926 15:08:22.185655 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0926 15:08:22.205256 (+ 19601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0926 15:08:22.205262 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0926 15:08:22.205720 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0926 15:08:22.205945 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:22.205961 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.211244 (+ 5283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.211250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:22.212679 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0926 15:08:22.212697 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.213733 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.213740 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0926 15:08:22.214011 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:22.214551 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0926 15:08:22.214575 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0926 15:08:22.218060 (+ 3485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0926 15:08:22.236225 (+ 18165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:22.236234 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:22.236239 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:22.236883 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:22.237536 (+ 653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0926 15:08:22.237542 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0926 15:08:22.237547 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0926 15:08:22.237738 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0926 15:08:22.243860 (+ 6122us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0926 15:08:22.243867 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0926 15:08:22.244089 (+ 222us) client_negotiation.cc:769] Sending connection context
0926 15:08:22.244264 (+ 175us) client_negotiation.cc:240] Negotiation successful
0926 15:08:22.244629 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223}
I20260926 15:08:22.246567 312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:22.185846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46140 (local address 127.9.123.124:36169)
0926 15:08:22.202924 (+ 17078us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:22.202929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:22.202950 (+ 21us) server_negotiation.cc:407] Connection header received
0926 15:08:22.203004 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:22.203008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:22.203089 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:22.206093 (+ 3004us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:22.206358 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.207531 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.212862 (+ 5331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.213565 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.214319 (+ 754us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:22.225362 (+ 11043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:22.225369 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:22.225375 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:22.225415 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:22.235153 (+ 9738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:22.237086 (+ 1933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:22.237092 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:22.237156 (+ 64us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:22.237381 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:22.242325 (+ 4944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:22.242331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:22.242335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:22.242776 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:22.245631 (+ 2855us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:22.245797 (+ 166us) server_negotiation.cc:299] Negotiation successful
0926 15:08:22.246069 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16933,"thread_start_us":17332,"threads_started":1}
I20260926 15:08:22.249763 32736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:22.183440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46124 (local address 127.9.123.124:36169)
0926 15:08:22.183617 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:22.183623 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:22.201543 (+ 17920us) server_negotiation.cc:407] Connection header received
0926 15:08:22.203375 (+ 1832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:22.203380 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:22.203452 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:22.203517 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:22.209393 (+ 5876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.210409 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.229257 (+ 18848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:22.230079 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:22.230255 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:22.237969 (+ 7714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0926 15:08:22.237978 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0926 15:08:22.237983 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0926 15:08:22.238023 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0926 15:08:22.242122 (+ 4099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:22.247119 (+ 4997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:22.247125 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:22.247128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:22.247303 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0926 15:08:22.247841 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0926 15:08:22.247846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0926 15:08:22.247850 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0926 15:08:22.248234 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0926 15:08:22.248489 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:22.249151 (+ 662us) server_negotiation.cc:299] Negotiation successful
0926 15:08:22.249430 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260926 15:08:22.252689 32659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16a0833d0b5a46a3b6db5a4ee69e66bc: Bootstrap starting.
I20260926 15:08:22.256675 32659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16a0833d0b5a46a3b6db5a4ee69e66bc: Neither blocks nor log segments found. Creating new log.
I20260926 15:08:22.258486 32659 log.cc:824] T 00000000000000000000000000000000 P 16a0833d0b5a46a3b6db5a4ee69e66bc: Log is configured to *not* fsync() on all Append() calls
I20260926 15:08:22.263525 32659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16a0833d0b5a46a3b6db5a4ee69e66bc: No bootstrap required, opened a new log
I20260926 15:08:22.276051 310 mvcc.cc:205] Tried to move back new op lower bound from 7333622997794406400 to 7333622997612253184. Current Snapshot: MvccSnapshot[applied={T|T < 7333622997794406400}]
I20260926 15:08:22.275878 32742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24ef1a30fdec4655a65300e0e6bf2619 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14e19504a2ad4639a3c9f3ce785e8176. Latest consensus state: current_term: 1 leader_uuid: "14e19504a2ad4639a3c9f3ce785e8176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14e19504a2ad4639a3c9f3ce785e8176" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39155 } } peers { permanent_uuid: "16a0833d0b5a46a3b6db5a4ee69e66bc" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } } peers { permanent_uuid: "24ef1a30fdec4655a65300e0e6bf2619" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } } }
I20260926 15:08:22.276237 32742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24ef1a30fdec4655a65300e0e6bf2619 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:08:22.278606 313 mvcc.cc:205] Tried to move back new op lower bound from 7333622997794406400 to 7333622997612253184. Current Snapshot: MvccSnapshot[applied={T|T < 7333622997794406400}]
I20260926 15:08:22.288250 32742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24ef1a30fdec4655a65300e0e6bf2619 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "14e19504a2ad4639a3c9f3ce785e8176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14e19504a2ad4639a3c9f3ce785e8176" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39155 } } peers { permanent_uuid: "16a0833d0b5a46a3b6db5a4ee69e66bc" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } } peers { permanent_uuid: "24ef1a30fdec4655a65300e0e6bf2619" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } } }
I20260926 15:08:22.288547 32742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24ef1a30fdec4655a65300e0e6bf2619 [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:08:22.289223 32756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "14e19504a2ad4639a3c9f3ce785e8176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14e19504a2ad4639a3c9f3ce785e8176" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39155 } } peers { permanent_uuid: "16a0833d0b5a46a3b6db5a4ee69e66bc" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } } peers { permanent_uuid: "24ef1a30fdec4655a65300e0e6bf2619" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } } }
I20260926 15:08:22.289741 32756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176 [sys.catalog]: This master's current role is: LEADER
I20260926 15:08:22.293537 305 catalog_manager.cc:1347] Generated new cluster ID: edc931d4488845829e1b43d01c0e9fb0
I20260926 15:08:22.293649 305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260926 15:08:22.295769 32659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16a0833d0b5a46a3b6db5a4ee69e66bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14e19504a2ad4639a3c9f3ce785e8176" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39155 } } peers { permanent_uuid: "16a0833d0b5a46a3b6db5a4ee69e66bc" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } } peers { permanent_uuid: "24ef1a30fdec4655a65300e0e6bf2619" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } }
I20260926 15:08:22.296185 32659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16a0833d0b5a46a3b6db5a4ee69e66bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16a0833d0b5a46a3b6db5a4ee69e66bc, State: Initialized, Role: FOLLOWER
I20260926 15:08:22.296715 32659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16a0833d0b5a46a3b6db5a4ee69e66bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14e19504a2ad4639a3c9f3ce785e8176" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39155 } } peers { permanent_uuid: "16a0833d0b5a46a3b6db5a4ee69e66bc" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } } peers { permanent_uuid: "24ef1a30fdec4655a65300e0e6bf2619" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } }
I20260926 15:08:22.303778 318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16a0833d0b5a46a3b6db5a4ee69e66bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14e19504a2ad4639a3c9f3ce785e8176" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39155 } } peers { permanent_uuid: "16a0833d0b5a46a3b6db5a4ee69e66bc" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } } peers { permanent_uuid: "24ef1a30fdec4655a65300e0e6bf2619" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } } }
I20260926 15:08:22.304211 318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16a0833d0b5a46a3b6db5a4ee69e66bc [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:08:22.306088 32659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16a0833d0b5a46a3b6db5a4ee69e66bc [sys.catalog]: configured and running, proceeding with master startup.
I20260926 15:08:22.306527 32633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16a0833d0b5a46a3b6db5a4ee69e66bc [term 0 FOLLOWER]: Advancing to term 1
I20260926 15:08:22.306982 32634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "14e19504a2ad4639a3c9f3ce785e8176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "16a0833d0b5a46a3b6db5a4ee69e66bc"
I20260926 15:08:22.311376 32633 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 16a0833d0b5a46a3b6db5a4ee69e66bc [term 1 FOLLOWER]: Refusing update from remote peer 14e19504a2ad4639a3c9f3ce785e8176: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260926 15:08:22.319754 32634 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 16a0833d0b5a46a3b6db5a4ee69e66bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14e19504a2ad4639a3c9f3ce785e8176 in term 1.
I20260926 15:08:22.323318 301 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16a0833d0b5a46a3b6db5a4ee69e66bc" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260926 15:08:22.331734 305 catalog_manager.cc:1370] Generated new certificate authority record
I20260926 15:08:22.333719 305 catalog_manager.cc:1504] Loading token signing keys...
I20260926 15:08:22.361423 305 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176: Generated new TSK 0
I20260926 15:08:22.362684 305 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260926 15:08:22.368141 318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16a0833d0b5a46a3b6db5a4ee69e66bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14e19504a2ad4639a3c9f3ce785e8176. Latest consensus state: current_term: 1 leader_uuid: "14e19504a2ad4639a3c9f3ce785e8176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14e19504a2ad4639a3c9f3ce785e8176" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39155 } } peers { permanent_uuid: "16a0833d0b5a46a3b6db5a4ee69e66bc" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } } peers { permanent_uuid: "24ef1a30fdec4655a65300e0e6bf2619" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } } }
I20260926 15:08:22.368494 318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16a0833d0b5a46a3b6db5a4ee69e66bc [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:08:22.368528 301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "14e19504a2ad4639a3c9f3ce785e8176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14e19504a2ad4639a3c9f3ce785e8176" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39155 } } peers { permanent_uuid: "16a0833d0b5a46a3b6db5a4ee69e66bc" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } } peers { permanent_uuid: "24ef1a30fdec4655a65300e0e6bf2619" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } } }
I20260926 15:08:22.368815 301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14e19504a2ad4639a3c9f3ce785e8176 [sys.catalog]: This master's current role is: LEADER
I20260926 15:08:22.368786 327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16a0833d0b5a46a3b6db5a4ee69e66bc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "14e19504a2ad4639a3c9f3ce785e8176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14e19504a2ad4639a3c9f3ce785e8176" member_type: VOTER last_known_addr { host: "127.9.123.126" port: 39155 } } peers { permanent_uuid: "16a0833d0b5a46a3b6db5a4ee69e66bc" member_type: VOTER last_known_addr { host: "127.9.123.125" port: 42053 } } peers { permanent_uuid: "24ef1a30fdec4655a65300e0e6bf2619" member_type: VOTER last_known_addr { host: "127.9.123.124" port: 36169 } } }
I20260926 15:08:22.369204 327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16a0833d0b5a46a3b6db5a4ee69e66bc [sys.catalog]: This master's current role is: FOLLOWER
I20260926 15:08:22.395272 334 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260926 15:08:22.456723 32739 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:22.453616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39864 (local address 127.9.123.126:39155)
0926 15:08:22.453791 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:22.453796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:22.454585 (+ 789us) server_negotiation.cc:407] Connection header received
0926 15:08:22.454892 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:22.454899 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:22.454994 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:22.455072 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:22.456324 (+ 1252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39864: BlockingRecv error: recv got EOF from 127.0.0.1:39864 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260926 15:08:22.459353 32740 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:22.453368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60136 (local address 127.9.123.125:42053)
0926 15:08:22.453549 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:22.453554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:22.457520 (+ 3966us) server_negotiation.cc:407] Connection header received
0926 15:08:22.457806 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:22.457812 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:22.457915 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:22.457988 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:22.459076 (+ 1088us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60136: BlockingRecv error: recv got EOF from 127.0.0.1:60136 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260926 15:08:22.463375 312 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:22.460946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46154 (local address 127.9.123.124:36169)
0926 15:08:22.461166 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:22.461172 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:22.461504 (+ 332us) server_negotiation.cc:407] Connection header received
0926 15:08:22.461778 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:22.461784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:22.461901 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:22.462016 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:22.463113 (+ 1097us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46154: BlockingRecv error: recv got EOF from 127.0.0.1:46154 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260926 15:08:22.500262 32740 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:22.497764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60152 (local address 127.9.123.125:42053)
0926 15:08:22.497963 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:22.497969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:22.498539 (+ 570us) server_negotiation.cc:407] Connection header received
0926 15:08:22.498783 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:22.498790 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:22.498951 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:22.499074 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:22.500018 (+ 944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60152: BlockingRecv error: recv got EOF from 127.0.0.1:60152 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260926 15:08:22.509941 32740 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:22.507810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60158 (local address 127.9.123.125:42053)
0926 15:08:22.508007 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:22.508013 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:22.508041 (+ 28us) server_negotiation.cc:407] Connection header received
0926 15:08:22.508124 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:22.508130 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:22.508226 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:22.508348 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:22.509561 (+ 1213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60158: BlockingRecv error: recv got EOF from 127.0.0.1:60158 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260926 15:08:22.526829 32740 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:22.524184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60166 (local address 127.9.123.125:42053)
0926 15:08:22.524389 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:22.524395 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:22.524966 (+ 571us) server_negotiation.cc:407] Connection header received
0926 15:08:22.525301 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:22.525307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:22.525409 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:22.525524 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:22.526551 (+ 1027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60166: BlockingRecv error: recv got EOF from 127.0.0.1:60166 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260926 15:08:22.542135 32740 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:22.539462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60168 (local address 127.9.123.125:42053)
0926 15:08:22.539722 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:22.539729 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:22.540366 (+ 637us) server_negotiation.cc:407] Connection header received
0926 15:08:22.540595 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:22.540602 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:22.540704 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:22.540836 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:22.541762 (+ 926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60168: BlockingRecv error: recv got EOF from 127.0.0.1:60168 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
W20260926 15:08:22.557061 32740 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:22.554530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60176 (local address 127.9.123.125:42053)
0926 15:08:22.554853 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:22.554860 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:22.555419 (+ 559us) server_negotiation.cc:407] Connection header received
0926 15:08:22.555625 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:22.555632 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:22.555734 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:22.555854 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:22.556832 (+ 978us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60176: BlockingRecv error: recv got EOF from 127.0.0.1:60176 (error 108)
Metrics: {"server-negotiator.queue_time_us":153}
W20260926 15:08:22.580178 32740 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:22.577470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60186 (local address 127.9.123.125:42053)
0926 15:08:22.577689 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:22.577696 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:22.578359 (+ 663us) server_negotiation.cc:407] Connection header received
0926 15:08:22.578637 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:22.578643 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:22.578755 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:22.578886 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:22.579887 (+ 1001us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60186: BlockingRecv error: recv got EOF from 127.0.0.1:60186 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260926 15:08:22.621210 32740 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:22.618435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60196 (local address 127.9.123.125:42053)
0926 15:08:22.618651 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:22.618658 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:22.619263 (+ 605us) server_negotiation.cc:407] Connection header received
0926 15:08:22.619520 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:22.619527 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:22.619659 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:22.619774 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:22.620738 (+ 964us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60196: BlockingRecv error: recv got EOF from 127.0.0.1:60196 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260926 15:08:22.690678 32740 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:22.688576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60208 (local address 127.9.123.125:42053)
0926 15:08:22.688785 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:22.688791 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:22.689129 (+ 338us) server_negotiation.cc:407] Connection header received
0926 15:08:22.689336 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:22.689343 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:22.689473 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:22.689592 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:22.690429 (+ 837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60208: BlockingRecv error: recv got EOF from 127.0.0.1:60208 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260926 15:08:22.824241 32740 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:22.822421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60216 (local address 127.9.123.125:42053)
0926 15:08:22.822684 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:22.822691 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:22.822880 (+ 189us) server_negotiation.cc:407] Connection header received
0926 15:08:22.823081 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:22.823086 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:22.823160 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:22.823248 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:22.824019 (+ 771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60216: BlockingRecv error: recv got EOF from 127.0.0.1:60216 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
I20260926 15:08:23.088677 32753 catalog_manager.cc:1259] Loaded cluster ID: edc931d4488845829e1b43d01c0e9fb0
I20260926 15:08:23.088793 32753 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 24ef1a30fdec4655a65300e0e6bf2619: loading cluster ID for follower catalog manager: success
I20260926 15:08:23.091777 32753 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 24ef1a30fdec4655a65300e0e6bf2619: acquiring CA information for follower catalog manager: success
W20260926 15:08:23.092849 32740 negotiation.cc:337] Failed RPC negotiation. Trace:
0926 15:08:23.090866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60218 (local address 127.9.123.125:42053)
0926 15:08:23.091073 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:23.091080 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:23.091428 (+ 348us) server_negotiation.cc:407] Connection header received
0926 15:08:23.091633 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:23.091638 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:23.091715 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:23.091802 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0926 15:08:23.092614 (+ 812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60218: BlockingRecv error: recv got EOF from 127.0.0.1:60218 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260926 15:08:23.094074 32753 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 24ef1a30fdec4655a65300e0e6bf2619: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 15:08:23.400032 334 catalog_manager.cc:1259] Loaded cluster ID: edc931d4488845829e1b43d01c0e9fb0
I20260926 15:08:23.400172 334 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 16a0833d0b5a46a3b6db5a4ee69e66bc: loading cluster ID for follower catalog manager: success
I20260926 15:08:23.403168 334 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 16a0833d0b5a46a3b6db5a4ee69e66bc: acquiring CA information for follower catalog manager: success
I20260926 15:08:23.405357 334 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 16a0833d0b5a46a3b6db5a4ee69e66bc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260926 15:08:23.616251 400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:23.609337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60224 (local address 127.9.123.125:42053)
0926 15:08:23.610154 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:23.610158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:23.610176 (+ 18us) server_negotiation.cc:407] Connection header received
0926 15:08:23.610251 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:23.610255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:23.610396 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:23.610501 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:08:23.611030 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:23.611811 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:23.613776 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:23.614261 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:23.614367 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:23.614970 (+ 603us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:08:23.615537 (+ 567us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:08:23.615685 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:23.615817 (+ 132us) server_negotiation.cc:299] Negotiation successful
0926 15:08:23.615894 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":501,"threads_started":1}
I20260926 15:08:23.628660 406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:23.622594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46168 (local address 127.9.123.124:36169)
0926 15:08:23.623235 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:23.623241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:23.623274 (+ 33us) server_negotiation.cc:407] Connection header received
0926 15:08:23.623378 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:23.623382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:23.623528 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:23.623684 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:08:23.624097 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:23.624816 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:23.626178 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:23.626590 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:23.626703 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:23.627437 (+ 734us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:08:23.627959 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:08:23.628115 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:23.628270 (+ 155us) server_negotiation.cc:299] Negotiation successful
0926 15:08:23.628352 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":393,"threads_started":1}
I20260926 15:08:23.653162 400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:23.634712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60236 (local address 127.9.123.125:42053)
0926 15:08:23.634934 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:23.634941 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:23.646315 (+ 11374us) server_negotiation.cc:407] Connection header received
0926 15:08:23.646423 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:23.646430 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:23.646533 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:23.646642 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:08:23.647590 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:23.648758 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:23.650867 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:23.651533 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:23.651631 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:23.652283 (+ 652us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:08:23.652443 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:08:23.652505 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:23.652730 (+ 225us) server_negotiation.cc:299] Negotiation successful
0926 15:08:23.652816 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260926 15:08:23.654486 413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:23.634919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39866 (local address 127.9.123.126:39155)
0926 15:08:23.645169 (+ 10250us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:23.645178 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:23.646924 (+ 1746us) server_negotiation.cc:407] Connection header received
0926 15:08:23.647212 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:23.647217 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:23.647434 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:23.647611 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:08:23.648390 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:23.649489 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:23.651521 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:23.652126 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:23.652230 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:23.652882 (+ 652us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:08:23.653377 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:08:23.653477 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:23.654102 (+ 625us) server_negotiation.cc:299] Negotiation successful
0926 15:08:23.654195 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10055,"thread_start_us":9964,"threads_started":1}
I20260926 15:08:23.659451 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 32518
I20260926 15:08:23.660197 406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0926 15:08:23.646896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46176 (local address 127.9.123.124:36169)
0926 15:08:23.647070 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0926 15:08:23.647075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0926 15:08:23.654134 (+ 7059us) server_negotiation.cc:407] Connection header received
0926 15:08:23.654399 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0926 15:08:23.654405 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0926 15:08:23.654500 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0926 15:08:23.654563 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0926 15:08:23.655210 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:23.656217 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:23.658300 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0926 15:08:23.658722 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0926 15:08:23.658841 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0926 15:08:23.659378 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0926 15:08:23.659520 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0926 15:08:23.659585 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0926 15:08:23.659816 (+ 231us) server_negotiation.cc:299] Negotiation successful
0926 15:08:23.659906 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260926 15:08:23.660485 32580 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:08:23.806457 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 32518
I20260926 15:08:23.823660 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 32592
I20260926 15:08:23.824738 32654 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:08:23.930927 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 32592
I20260926 15:08:23.947309 9709 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskR962dp/build/asan/bin/kudu pid 32666
I20260926 15:08:23.948477 32728 generic_service.cc:196] Checking for leaks (request via RPC)
I20260926 15:08:24.054982 9709 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskR962dp/build/asan/bin/kudu with pid 32666
2026-09-26T15:08:24Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4829 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9650 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (364917 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST